Current time: 01-28-2021, 07:57 PM Hello There, Guest! (LoginRegister)

Post Reply 
 
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Database Query Search Time
08-24-2010, 11:52 PM
Post: #41
RE: Database Query Search Time
Travis,

First, great post! I have read through it 3 times and there is a ton of great info here.

I am upgrading a clients verity collection to solr, and I am having many woes, most notably the dreaded exceeded maxWarmingSearchers. I am the guy that posted this issue on coldfusionjedi
link to post.

Deleting and re-creating the collection did help the issue initially, but when indexing large numbers of files, I think its unavoidable, and you are forced to dramatically slow down your indexing to allow the searchers to catch up.

I entered this as a bug and it has been confirmed, with a workaround suggested. One way is to do a bulk insert, they way you are doing it, which will not work for indexing physical files where there is no way to know the body ahead of time, the other is the way I am currently doing it, which is to schedule small batches (4 files since my maxSearchWarmers is set to 4) every 90 seconds or so. As you can guess indexing 60k+ files is going to take a very long time. (300+ hours?)

Increasing maxSearchWarmers and increasing the number of files only moves the point at which the error occurs and uses more resources.

I am looking for a fresh perspective on the issue, and perhaps hoping that the knowledge you have gleaned from your research may permit you to offer a suggestion.

Thanks!
Anthony DeSalvatore
Find all posts by this user
Quote this message in a reply
Post Reply 


Messages In This Thread
RE: Database Query Search Time - pmeenan - 07-04-2010, 12:03 PM
RE: Database Query Search Time - pmeenan - 07-05-2010, 04:35 AM
RE: Database Query Search Time - sajal - 07-05-2010, 05:03 AM
RE: Database Query Search Time - pmeenan - 07-08-2010, 08:47 PM
RE: Database Query Search Time - sajal - 08-13-2010, 10:06 PM
RE: Database Query Search Time - sajal - 08-14-2010, 03:38 AM
RE: Database Query Search Time - sajal - 08-14-2010, 06:50 AM
RE: Database Query Search Time - pmeenan - 08-16-2010, 02:55 AM
RE: Database Query Search Time - sajal - 08-17-2010, 01:14 AM
RE: Database Query Search Time - pmeenan - 08-23-2010, 01:22 AM
RE: Database Query Search Time - pmeenan - 08-23-2010, 08:45 PM
RE: Database Query Search Time - ringman57 - 08-24-2010 11:52 PM
RE: Database Query Search Time - ringman57 - 08-25-2010, 02:54 AM

Forum Jump:


User(s) browsing this thread: 2 Guest(s)