我正在使用Apache Lucene库来创建我的网站的搜索功能.该网站正在从Sharepoint RSSFeed获取其所有内容,因此每次我都必须通过所有的RSSFeed网址并阅读内容.为了使搜索功能更快,我创建了一个计划任务,每隔一小时进行一次索引:
<bean id="rssIndexerService" class="com.lloydsbanking.webmi.service.RSSIndexerService" />
<task:scheduled-tasks> <task scheduled ref="rssIndexerService" method="indexUrls" cron="0 0 * * * MON-FRI" /></task:scheduled-tasks>
问题是,如果我创建一个新的内容,那么在服务器运行时,搜索不会显示新的内容,并且在调用计划任务后,如果我删除一个条目,它仍然不会显示从索引文件中删除的条目。以下是索引代码:
@Service
public class RSSIndexerService extends RSSReader {
@Autowired
private RSSFeedUrl rssFeedUrl;
private IndexWriter indexWriter = null;
private String indexPath = "C:\\MI\\index";
Logger log = Logger.getLogger(RSSIndexerService.class.getName());
public void indexUrls() throws IOException {
Date start = new Date();
IndexWriter writer = getIndexWriter();
log.info("Reading all the Urls in the Sharepoint");
Iterator<Entry<String, String>> entries = rssFeedUrl.getUrlMap().entrySet().iterator();
try {
while (entries.hasNext()) {
Entry<String, String> mapEntry = entries.next();
String url = mapEntry.getValue();
SyndFeed feed = rssReader(url);
for (Object entry : feed.getEntries()) {
SyndEntry syndEntry = (SyndEntry) entry;
SyndContent desc = syndEntry.getDescription();
if (desc != null) {
String text = desc.getValue();
if ("text/html".equals(desc.getType())) {
Document doc = new Document();
text = extractText(text);
Field fieldTitle = new StringField("title", syndEntry.getTitle(), Field.Store.YES);
doc.add(fieldTitle);
Field pathField = new StringField("path", url, Field.Store.YES);
doc.add(pathField);
doc.add(new TextField("contents", text, Field.Store.YES));
// New index, so we just add the document (no old document can be there):
writer.addDocument(doc);
}
}
}
}
} finally {
// closeIndexWriter();
}
Date end = new Date();
log.info(end.getTime() - start.getTime() + " total milliseconds");
}
public IndexWriter getIndexWriter() throws IOException {
if (indexWriter == null) {
Analyzer analyzer = new StandardAnalyzer(Version.LUCENE_47);
log.info("Indexing to directory '" + indexPath + "'...");
Directory dir = FSDirectory.open(new File(indexPath));
IndexWriterConfig config = new IndexWriterConfig(Version.LUCENE_47, analyzer);
config.setOpenMode(OpenMode.CREATE_OR_APPEND);
indexWriter = new IndexWriter(dir, config);
}
return indexWriter;
}
@PreDestroy
public void closeIndexWriter() throws IOException {
if (indexWriter != null) {
System.out.println("Done with indexing ...");
indexWriter.close();
}
}
}
我知道问题可能是由config.setOpenMode(OpenMode.CREATE_OR_APPEND)引起的;,但我不知道该怎么解决。
好吧,我想到了检查目录之前是否为空的想法,如果不是,则删除之前的索引,然后每次在OpenModel. Create中进行索引:
File path = new File(System.getProperty("java.io.tmpdir")+"\\index");
Directory dir = FSDirectory.open(path);
Analyzer analyzer = new StandardAnalyzer(Version.LUCENE_47);
IndexWriterConfig config = new IndexWriterConfig(Version.LUCENE_47, analyzer);
if (path.list() != null) {
log.info("Delete previous indexes ...");
FileUtils.cleanDirectory(path);
}
config.setOpenMode(OpenMode.CREATE);
然后我简单的使用addDocument():
if ("text/html".equals(desc.getType())) {
...
// New index, so we just add the document (no old document can be there):
indexWriter.addDocument(doc);
}