Database Query Search Time

[green-watch.org]

Greetings Anthony,

When I started having my maxWarmingSearchers problem, I actually came across your post on coldfusionjedi. I purged my collection and then reindexed it all over again like you did. It worked great at first and the reason why is because data takes less time to index with smaller collections.

Indexing takes longer as a SOLR collection grows larger. With my first indexing attempt, my script started executing while another one was already in progress. That really bogged down the server and things got messed up rather quickly. At that point, I created a variable in my GLOBAL table that checked to see if the indexing script was already in progress before executing the indexing code again. Having said that, even if the variable indicated the code is not currently executing, it took some time before the number of files got recognized by the coldfusion collection. I think there is some sort of "commit" bug if files get indexed too quickly.

You might want to revisit the bulk insert concept. If you have actual files, you can use CFFILE to read the content into a variable, and then use that variable to create a CFQUERY row. Once you have the CFQUERY created, you can pass that to the CFINDEX to do the bulk insert.

I wish I could see what goes on under the hood of CFINDEX. I am sure it would be possible to create a custom function like i did for the CFSEARCH function.

By the way, if you want to post some example indexing code, I can take a glance at it on here and see if I have any suggestions.

Sincerely,
Travis Walters