Remove lines from updating csv file

Question

In my wamp server I have a script that updaing csv file every second. I want to remove lines from the csv file, I have this code (from here: How To Delete The Top 100 Rows From a CSV File With PHP):

$input = explode("\n", file_get_contents("file.csv"));
foreach ($input as $line) {
 // process all lines.
}

// This function removes first 100 elements.
// More info:
// http://php.net/manual/en/function.array-slice.php
$output = array_slice($input, 100);
file_put_contents("out.csv", implode("\n", $output));

The problem is happened when I trying to run the above code and I get:

"failed to open stream permission denied"

The problem is because the file is updating all the time. How I know that? I mada a copy of the csv (the content of this file is not updating at all) and it's success.

Why I need this?

Because the file is growing I have to remove lines from him..if I don't do this I get: "PHP Fatal error: Allowed memory size of ******** bytes exhausted (tried to allocate 71 bytes) in /var/www/*******.php on line 54," even if I write ini_set("memory_limit", "-1");

I remind you - it's wamp server!

Any idea?

Thanks!

Answer 1

$input = explode("\n", file_get_contents("file.csv"));
foreach ($input as $line) {
 // process all lines.
}

// This function removes first 100 elements.
// More info:
// http://php.net/manual/en/function.array-slice.php
$output = array_slice($input, 100);
file_put_contents("out.csv", implode("\n", $output));

The problem is happened when I trying to run the above code and I get:

"failed to open stream permission denied"

You have 3 cases here where you can't modify the file.

1. File Already in Use (Locked by another process).
2. File does not have the right permissions (R/W). Or file does not belong to the user/group you are using, in must of php cases it's (www-data).
3. File does not exist.

1) If the file is already in use, you can't modify it, but you can try something else * A RACE CONDITION * that can make it happen :

# RACE CONDITION LOOP
# The only way you have here is to setup a RC loop to check if/when the file can be written.
$Path   = dirname ( __FILE__ ) . "path_to_your_file/out.csv";
$tries  = 99; // Any number of tries you want.
for ( $i = 1; $i < $tries; $i++ )
{
    $output = array_slice ( $input, 100 );
    {
        if ( @file_put_contents ( $Path , implode ( "\n", $output ) ) )
        {
            echo "Content Added!"."<br>";
            $i = $tries;
        }
        else
        {
            echo "Content Cannot be Added!"."<br>";
        }
    }

    ob_flush ( );
    echo "We Have tried " . $i . " times to add content to the file"."<br>";
    sleep ( 1 );
    flush ( );
}

2) If File does not have the right permissions you can set it in your code :

$Path   = dirname ( __FILE__ ) . "path_to_your_file/out.csv";
$Mode   = 0755; // permissions wanted
$Umsk   = @umask( 0 );
@chmod ( $Path, $Mode );
@umask ( $Umsk );

3) If File does not exist you can create it, but first make sure to check if it's there :

# Make sure the file that contain this code have the right permissions, user/group .
# If not this cannot create the file and will not allow you execute the whole code .
$Path   = dirname ( __FILE__ ) . "path_to_your_file/out.csv";
$Mode   = 0755; // permissions wanted
$Check_File = file_exists ( $Path );
if ( $Check_File ) { $File_Exist = true; }
else { $File_Exist = false; }

if ( $File_Exist )
{
    @touch ( $Path );
    @chmod ( $Path, $Mode );
}
else
{
    echo "File Already Exist";
}

I hope this can help and if not please specify what exactly the problem is.

PS : for memory limit errors some servers and/or host provider does not allow this in user's php.ini or from php to be set. you must edit the global config files for both php and apache, and maybe mysql too in case you are using big amount of data for database, to allow a max chosen value for memory limit .

Answer 2

It's not what you asked. But in my opinion your existing scheme is faulty. And then you are looking for a silly solution to fix it. You have multi-task issues and it is not a fast process. You will also probably lose a few lines every time you do this.

The proper way to do this is to have a folder say LOGS.

Then put a file in of the name YYYYMMDD.CSV. Change the name every day. And delete any that are say, over 7 days old.