RAM capping while downloading large files

[melkiik]

Hi fellow techspotters,

odd thing has happened to my friend, whos pc I have handbuilt. Every time he starts downloading lets say 1gb or bigger file his RAM starts to fill up. No processes are visible to be the cause. Cpu usage stays on normal level, but the ram just keeps on filling. Until it is throttled by capping on 8gb and he is forced to pause the download and reboot the machine, and the RAM is cleaned.

I posted this under networking, because I doubt that it is a hardware related problem.
The numbers go some like this. 8gb of RAM = 20gb of download -> cap -> restart -> ram clear -> restore download. And the RAM gets used oddly like its ment to behave like this. Does it store something on the way?

It isnt actually a problem, my friend can cope with it. But I as a builder of the PC would really like to solve this. I have never encountered such memory behavior with downloads before.

Screencaps of the situation available when I get home from work.

cheers
m

 

[cliffordcooley]

Does the machine have an anti-virus installed? And if so, does memory usage have the same effect with or without active scanning?

When monitoring Task Manager, which process is using the memory allotment that appears to be accumulating? Killing the process should clear memory usage by that process.

Forgive me as I beat around the bush, not really knowing what to say.

 

[melkiik]

"Forgive me as I beat around the bush, not really knowing what to say."

Exactly how I feel.

No there is no active antiv. program. Though we installed one and ran a scan for malware and viruses just incase, none found and uninstalled. And no, there is no process that could be allocated with the RAM usage.

This really is weird. I've gone through all the processes while he downloads, and also monitored his router incase there is some kind of buffer, but nothing.

 

[mailpup]

I posted this under networking, because I doubt that it is a hardware related problem.

You posted this under networking which is still under hardware. Just saying.

 

[melkiik]

You posted this under networking which is still under hardware. Just saying.

Damn I'm good.

 

[melkiik]

Okay, so I'm back with screenshots.

1st one shows all running processes during the download from highest mem to lowest.



2nd there is how much of the RAM is used atm, and on the left you can see other usages.



CPU normal compared to processes, and so is disk writing.
But the ethernet goes almost hand to hand with the RAM. Which is the case.

Why does the RAM fill up to the cap while downloading? Also downloading smaller files result in the RAM getting for example + 2gb used or whatever the size of the file. Before the computer getting restarted.

We have also tried changing RAM placement on the motherboard. >Zero results.
Also few programs that clear RAM, and re allocate it.
Changing timings, overclocking, undeclocking. No effect.

Is this really a one of a kind case?

Cheers

m

 

[melkiik]

I'm bumping this thread up 3rd time, sorry, but we are really looking for the solution! Anyone got any leads or ideas? Or even want to guess with me what is up?

There is also ongoing topics at toms and ocnet, but nobody seems to know the reason for such event. Which is really odd.

What is this sorcery, illuminati? *tinfoil

 

[DelJo63]

Until it is throttled by capping on 8gb and he is forced to pause the download and reboot the machine, and the RAM is cleaned.

hmm; WHY? Is there a symptom forcing the restart, or is it just fear - - this matters.
If there's some symptom (freeze) then that's an issue. Lag, slow response to switching windows or launching a new application IS NOT AN ERROR under this condition (hi ram utilization). This what ram is for - - to be consumed as needed. As long as the system has a good size pagefile setting, the task will run to completion and ram will collapse as expected.

btw: AVGUARD pid 1612 is the large memory program and it IS an a/v program
(see avguard)

In the Taskmgr, view->select columns

• memory - Working set, Working Set Delta, Commit Size

here's mine:

Working set will tend to force ram usage but that's ok
Delta shows activity in/out of the pagefile and more of this is why we see slowness as noted above.

 

[melkiik]

Thank you for the answer Jobeard.

We have looked into his pagefile and to the following columns. Ran a few test downloads to see what really goes down in there. But it remains mystery. Everything is exactly the way it is supposed to be. So this ghost data that hangs in the RAM really does not show anywhere. We have also used few RAM benchmark tools to determine the source but nothing.

After fully focused on the RAM, we decided that it can't be the RAM. RAM in this case is more of a symptom than a cause. Now we have traced the cause to be either SSD or HD.

First we tried downloading 60gb to a HD and it didn't fill the RAM while downloading.
Second we did the same on SSD and yes, the RAM again started to slowly reach its capacity.
Conclusion was ofcourse that SSD is the cause.

We double checked and suddenly it was the other way around!? This sorcery.
So we ran few more tests. Disconnect drives. Plugged in one at a time. Formatted drives, and tried again. No dice.

But the question is; is it possible that his connection speed overloads SSD or HDD write and unpacking speed so the DISK has to dump it's memory little by little to the RAM to keep up with the writing process. And can RAM even store active packets which are downloaded atm?
And I could belive this incase of the HDD which is a Caviar green, known for it's slow speeds, but in the case of SSD, seems a bit odd. That a 150mb/ broadband, DL 15mb/s roughly, would overrun a 500mb/s writespeed.

Any thoughts?

 

[melkiik]

And yes the RAM is the cause of restart, because of getting capped and requesting for one. The user does not need to boot the computer, it does it on it's own.

And yes AVGUARD was on, during the screenshot

 

[DelJo63]

I/O buffering: The whole point of buffering is to allow the processor to ignore the 'weakest ( aka slowest) link in the chain'. Take an input stream -> push to a process -> emit some results. Without buffering, processing crawls along using one-in, one-out and the results are then limited by how fast the inputs arrive. If we have a slow output device, it's never seen/felt as a limitation. Add high speed inputs targeted to a slow speed output device and the results still look/feel good *if* the output buffering can handle the excess. HD drivers have to queue requests (aka buffer) OR stall the process making the I/O request until it can be queued. Normally, even with queued writes, the buffers are still in the application to avoid useless mem-mem transfers.

packets which are downloaded atm

do you mean async transfer mode? Not frequently implemented unless you're using a TOR. Packet reassembly does require buffering and there's always the case that a missing packet holds up the process until it arrives.

The user does not need to boot the computer, it does it on it's own.

I think you're saying that the RAM issue actually induces system restart if allowed to continue.

To do this right, we would need the debug versions of the processes and to trace it while running - - ZERO probability heh?

It's difficult to determine the actual root-cause of this symptom - - we just don't have the necessary tools.
We're left with attempting to avoid / circumvent the issue.

Recommendations:

• avoid TOR as a source access, use simpler techniques like FTP or browser downloads.
• ensure SACKopts and TCP3123 are active on your tcp stack (both files attached will set these)

 

[melkiik]

I/O buffering: The whole point of buffering is to allow the processor to ignore the 'weakest ( aka slowest) link in the chain'. Take an input stream -> push to a process -> emit some results. Without buffering, processing crawls along using one-in, one-out and the results are then limited by how fast the inputs arrive. If we have a slow output device, it's never seen/felt as a limitation. Add high speed inputs targeted to a slow speed output device and the results still look/feel good *if* the output buffering can handle the excess. HD drivers have to queue requests (aka buffer) OR stall the process making the I/O request until it can be queued. Normally, even with queued writes, the buffers are still in the application to avoid useless mem-mem transfers.

do you mean async transfer mode? Not frequently implemented unless you're using a TOR. Packet reassembly does require buffering and there's always the case that a missing packet holds up the process until it arrives.
I think you're saying that the RAM issue actually induces system restart if allowed to continue.

To do this right, we would need the debug versions of the processes and to trace it while running - - ZERO probability heh?

It's difficult to determine the actual root-cause of this symptom - - we just don't have the necessary tools.
We're left with attempting to avoid / circumvent the issue.

Recommendations:

• avoid TOR as a source access, use simpler techniques like FTP or browser downloads.
• ensure SACKopts and TCP3123 are active on your tcp stack (both files attached will set these)

Hey,

sorry for the delay in replying, was away on business trip and finally back.

The problem actually got solved on its own, after few RAM calbrations. We underclocked the sticks my friend has 1600 to 1555mhz and the problem never occured again.

These things are sometimes pretty weird. You look for the solution everywhere and the answer is the most unlikely. Still we can't determine what is the cause. But obviously it seems to be the RAM after all, or atleast it's the only one running on different settings now.

But many thanks for your help and replys
My friend also set the files you gave us to the reg.

best regards

m