Processing Huge Data In Memory While within The Loop

Never process a huge data input(i.e data input greater than 30% of the maximum memory limit) at one time. Whenever possible, try to chunk the processing to more acceptable levels.

Take the following example.

$arr =  file("somehugefile.txt"); //buffers the array into memory
 foreach($arr as $line) {
    //do something
 }
 unset($arr);

Chunk the processing by, splitting the file to more manageable pieces.

$arr1 =  file("chunk1.txt"); //buffers the array into memory
 foreach($arr1 as $line) {
    //do something
 }
 unset($arr1);
 $arr2 = file("chunk2.txt"); //buffers the array into memory 
 foreach($arr2 as $line) {
    //do something
 }
 unset($arr2);
 $arr3 = file("chunk3.txt"); //buffers the array into memory
 foreach($arr3 as $line) {
    //do something
 }
 unset($arr3);

Related Links:

• Related Links Widget for Blogspot