我第一次尝试Lucene.Net
我有一个包含500个文档(html和pdf)的索引,其中包含一些字段,如url、内容、标题
当我在内容和/或标题中搜索时一切正常
但当我搜索URL时,我没有结果
所以我找到了像“/tlfdi/ep是/datenschutz2016/files/资产/公共/下载/page0004. pdf”这样的网址,但没有“page0004.pdf”
也有“*”它不起作用。
索引和搜索使用WhitespaceAnalyzer。使用Standard ardAnalyzer,当我搜索“/kontakt/index. aspx”时,我得到的结果为零
搜索:
WhitespaceAnalyzer analyzer = new WhitespaceAnalyzer();
MultiFieldQueryParser parser = new MultiFieldQueryParser(Version.LUCENE_30,
new[] { "url", "title", "description", "content", "keywords" }, analyzer);
Query query = parseQuery(searchQuery, parser);
Lucene.Net.Search.ScoreDoc[] hits = (Lucene.Net.Search.ScoreDoc[])searcher.Search(
query, null, hits_limit, Sort.RELEVANCE).ScoreDocs;
有人能帮忙吗?
分析器的标准票价不会做你想要的,相反,你必须编写一个自定义标记器和分析器。
很简单!我们只需要划掉一个标记器和一个分析器。
UrlTokenizer负责生成令牌。
// UrlTokenizer delimits tokens by whitespace, '.' and '/'
using AttributeSource = Lucene.Net.Util.AttributeSource;
public class UrlTokenizer : CharTokenizer
{
public UrlTokenizer(System.IO.TextReader @in)
: base(@in)
{
}
public UrlTokenizer(AttributeSource source, System.IO.TextReader @in)
: base(source, @in)
{
}
public UrlTokenizer(AttributeFactory factory, System.IO.TextReader @in)
: base(factory, @in)
{
}
//
// This is where all the magic happens for our UrlTokenizer!
// Whitespace, forward slash or a period are a token boundary.
//
protected override bool IsTokenChar(char c)
{
return !char.IsWhiteSpace(c) && c != '/' && c != '.';
}
}
UrlAnalyzer使用输入流并实现UrlTokenizer和LowerCaseFilter以进行不区分大小写的搜索。
// Custom Analyzer implementing UrlTokenizer and LowerCaseFilter.
public sealed class UrlAnalyzer : Analyzer
{
public override TokenStream TokenStream(System.String fieldName, System.IO.TextReader reader)
{
//
// This is where all the magic happens for UrlAnalyzer!
// UrlTokenizer token text are filtered to lowercase text.
return new LowerCaseFilter(new UrlTokenizer(reader));
}
public override TokenStream ReusableTokenStream(System.String fieldName, System.IO.TextReader reader)
{
Tokenizer tokenizer = (Tokenizer)PreviousTokenStream;
if (tokenizer == null)
{
tokenizer = new UrlTokenizer(reader);
PreviousTokenStream = tokenizer;
}
else
tokenizer.Reset(reader);
return tokenizer;
}
}
SO这是演示UrlAnalyzer的代码。为了清晰起见,我用QueryParser替换了MultiFieldQueryParser。
//
// Demonstrate UrlAnalyzer using an in memory index.
//
public static void testUrlAnalyzer()
{
string url = @"/tlfdi/epapers/datenschutz2016/files/assets/common/downloads/page0004.pdf";
UrlAnalyzer analyzer = new UrlAnalyzer();
Directory directory = new RAMDirectory();
QueryParser parser = new QueryParser(Lucene.Net.Util.Version.LUCENE_30, "url", analyzer);
IndexWriter writer = new IndexWriter(directory, analyzer, true, new IndexWriter.MaxFieldLength(2048));
//
// index a document. We're only interested in the "url" field of a document.
//
Document doc = new Document();
Field field = new Field("url", url, Field.Store.NO, Field.Index.ANALYZED);
doc.Add(field);
writer.AddDocument(doc);
writer.Commit();
//
// search the index for any documents having 'page004.pdf' in the url field.
//
string searchText = "url:page0004.pdf";
IndexReader reader = IndexReader.Open(directory, true);
IndexSearcher searcher = new IndexSearcher(reader);
Query query = parser.Parse(searchText);
ScoreDoc[] hits = searcher.Search(query, null, 10, Sort.RELEVANCE).ScoreDocs;
if (hits.Length == 0)
throw new System.Exception("RamblinRose is fail!");
//
// search the index for any documents having the full URL we indexed.
//
searchText = "url:\"" + url + "\"";
query = parser.Parse(searchText);
hits = searcher.Search(query, null, 10, Sort.RELEVANCE).ScoreDocs;
if (hits.Length == 0)
throw new System.Exception("RamblinRose is fail!");
}
Lucene.net是好东西。我希望这段代码能提高你对Lucene分析的理解。
祝你好运!
p. s.当心通配符搜索来解决问题,它是大型索引的真正杀手。