ionCube Logo
 FAQFAQ   SearchSearch   MemberlistMemberlist   UsergroupsUsergroups   RegisterRegister 
 ProfileProfile   Log in to check your private messagesLog in to check your private messages   Log inLog in 


 
Post new topic   Reply to topic    ionCube Forum Index -> ionCube PHP Encoder

Excessivly high memory usage

Author Message
alainr



Joined: 15 Aug 2016
Posts: 5

PostPosted: Mon Aug 15, 2016 11:51 pm    Post subject: Excessivly high memory usage Reply with quote

Hi there,

We have a script that extracts a lot of data (a few million database rows) from ODBC databases in order to transfer between servers. When we run through a loop to pull through data, an unencoded version of our PHP file will use roughly 5MiB of PHP memory (according to
Code:
memory_get_usage(true)
), however this jumps to over 150MiB when using an encoded version of the same file.

The code looks roughly like

Code:

$output->data = array();
$result = odbc_exec(...);
$inline_utf8_encode = create_function('&$val, $key', 'if (!is_numeric($val)) $val = utf8_encode($val);');
while($row = odbc_fetch_array($result)) {
   array_walk($row, $inline_utf8_encode);
   $output->data['a']['b'][] = $row;
}
echo json_encode($output);


We updated to version 5.0.21 of the loader from v4 which made some difference, but not enough to prevent out-of-memory errors in PHP. We're seeing this memory problem when running on Windows 32bit VC9 PHP 5.3 with encoder 6.5 (since we still have PHP 5.2 sites that we need to support). Interestingly though, the memory problem does not happen on PHP 5.2.

Does anyone have any tips about how to decrease memory usage that's used by ionCube encoded files?[/code]
Back to top
View user's profile Send private message
alastair



Joined: 23 Feb 2010
Posts: 188

PostPosted: Tue Aug 16, 2016 1:13 pm    Post subject: Reply with quote

Hi,

Do you get the same problem if you use PHP 5 encoding with your version 8 Encoder ? Encoding with the PHP 5 version 8 Encoder will allow the files to run on PHP 5.0 upwards provided that a 4.4 or above Loader is installed.

If you do get the same problem with version 8 encoding then please create a ticket in our Support Help Desk at http://support.ioncube.com If possible, please create a self-contained test case that reproduces the problem; judging by the code snippet you have provided that should be feasible.
_________________
Alastair

ionCube
Back to top
View user's profile Send private message
alainr



Joined: 15 Aug 2016
Posts: 5

PostPosted: Tue Aug 16, 2016 11:02 pm    Post subject: Reply with quote

So after further experimentation using both the online encoder and command line versions of ioncube_encoder 6.5, 7.0, 8.3 and even 9.0, when specifying PHP 5.0 as the source language, the memory usage spikes.

We tried it with more data and were getting scripts that were using 40MiB of ram when unencoded, take over 1GiB of ram when encoded.

What was interesting though is that the memory usage does not spike when targeting 5.3 as the source language; generally we saw memory usage only increase by about 50% (e.g. 62MiB usage instead of 40MiB in the example above)

Frustratingly, our PHP 5.2 sites that we still support are using ioncube_loader 3.1.32, so using an encoder newer than 6.5 is problematic; which is also annoying since it's only our PHP 5.3 sites that are getting this high memory usage - and the latest loader crashes on PHP 5.2. I'll experiment with some of the version 4 loaders to see if any of these load without crashing.

Also frustrating, is that when we run an offending code block in isolation (we do about 30 different loops similar to the code example below, for different sets of data that need to be moved), it doesn't use as much memory as when it's in the full script.
Back to top
View user's profile Send private message
alainr



Joined: 15 Aug 2016
Posts: 5

PostPosted: Wed Aug 17, 2016 12:20 am    Post subject: Reply with quote

Ah, the reason PHP 5.2 was crashing with the latest loader for us was because we're supporting a VC6 version of PHP 5.2, but it works fine with the 4.0.7 loader
Back to top
View user's profile Send private message
alainr



Joined: 15 Aug 2016
Posts: 5

PostPosted: Thu Aug 18, 2016 7:06 am    Post subject: Reply with quote

So here's where it gets curly.

Our script is quite large, and once encoded (with the encoders with problems) ends up being a .php file over 256KiB

This 256KiB seems to be significant, as I can chop code out of this file to bring it down under this size and it will run without issue. I can also chop the large .php file into smaller .php files, all just including each other in a chain, and the exact same code that was leaking memory previously is no longer leaking.

Is there a known issue with ionCube with .php files over 256KiB in size?
Back to top
View user's profile Send private message
alastair



Joined: 23 Feb 2010
Posts: 188

PostPosted: Thu Aug 18, 2016 2:07 pm    Post subject: Reply with quote

Hi,

Thanks for the further information.

The size of the file should not matter in itself and we do not know of any issues of that type. So it is surprising that the issue appears to go away if the file is chopped into smaller files. What is the total size of the encoded versions of the smaller files?

The only known issue in this area is if an include (or require) occurs in a loop. Do you have any code like that?

On the create_function manual page it says that, "... it has bad performance and memory usage characteristics." However, that should not be any worse for the encoded version than the unencoded version.
_________________
Alastair

ionCube
Back to top
View user's profile Send private message
alainr



Joined: 15 Aug 2016
Posts: 5

PostPosted: Thu Aug 18, 2016 9:57 pm    Post subject: Reply with quote

The total size of the individual files (264KiB) is larger than the size of the individual file (257KiB).

No, there's no includes in any loops.

Further testing shows that it's definitely to do with the file size. Chopping code out of the file so that it brings it down to less than 256KiB makes the memory spikes magically go away. eg, if we chop a loop or two out of the end of the file, it'll make the memory leaks at the start of the file disappear.

Replacing the create_function/array_walk calls with a standard foreach loop also didn't help. It seems to have some loose relationship with the use of utf8_encode() though, if we're not using that the memory only climbs a little - but still more than if the output file is less than the magic 256KiB (80MiB of memory instead of 60MiB)

What we also found was that outputting both memory_get_usage(false) and memory_get_usage(true) helps track down when the problem is happening. When there's no memory leaks the results of these two functions are within a few kilobytes of memory of each other, when the memory leaks do occur they are wildly different.
Back to top
View user's profile Send private message
Display posts from previous:   
Post new topic   Reply to topic    ionCube Forum Index -> ionCube PHP Encoder All times are GMT + 1 Hour
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum