If you are experiencing high memory comsumption with Lucene 2.9.0/3.0.1 it could be due…
- ... to a recently reported and fixed bug in StandardTokenizer where JFlex generated code was expanding a buffer (zzBuffer) and never trimming it down
- ... to another recently reported and fixed bug where IndexWriter held references to Readers used in your Fields, (and if you have apache tika's reader, those can take up a lot of space)