Snowden used basic 'web crawler' software to scrape data from NSA

[Shawn Knight]

Edward Snowden’s treasure trove of classified National Security Agency documents has forever changed how the world views privacy. But as everyone got caught up in the ongoing leaks, nobody has publically questioned exactly how Snowden got his hands on the information – until now.

One would be forgiven to think that Snowden used some elaborate hacking software to siphon top secret documents during his time as a contractor for the agency. But truth be told, his methods were much more rudimentary.

According to a senior intelligence official, Snowden used ordinary “web crawler” software typically used to search, index and back up a website. The software ran in the background while he worked, scraping the agency’s systems and ultimately accessing an estimated 1.7 million files. Although it appears as though he did set specific search parameters such as which subjects to search for and how deeply to dig, it should have easily been detected.

Key to Snowden’s success was the fact that he was working for the agency in Hawaii, a location that had not yet been outfitted with the latest security measures.

Had he been at the NSA’s headquarters in Fort Meade, Maryland, he almost certainly would have been caught. Agency officials said systems at that location are constantly monitored and accessing / downloading a large volume of data would have been noticed.

Snowden’s behavior did draw attention a few times but officials say he was able to get off the hook by providing legitimate-sounding explanations for his activity.

Permalink to story.

https://www.techspot.com/news/55609-snowden-used-basic-web-crawler-software-to-scrape-data-from-nsa.html

 

[Guest]

I do not know that...

 

[oldikes]

I'd like to hear what explanations he used....Curious how in the world he got away with it after it getting noticed.

 

[Guest]

If Snowden used off the shelf webcrawler software and was able to install it in the NSA systems,then the military and the NSA is incompetent. I am pretty sure this is a cover story to explain how the impossible was accomplished without help. If not, then we should get our
tax dollars back from the NSA.

 

[MilwaukeeMike]

If Snowden used off the shelf webcrawler software and was able to install it in the NSA systems,then the military and the NSA is incompetent. I am pretty sure this is a cover story to explain how the impossible was accomplished without help. If not, then we should get our
tax dollars back from the NSA.

Yeah, they can put that refund on the same check as the healthcare.gov refund and the Solyndra refund.

 

[Renrew]

If you think our governments IT programs are sound or impervious to hacking you're living in Dreamland. An internal audit revealed that over 80 percent of all government websites have major problems with security. Some, such as Obamacare, have no security at all, even with 500 to 800 Million Taxpayer $ spent.

 

[SirGCal]

There is a reason the best hackers in the world are often offered jobs by such industries. Banks, IT companies, Government, etc. Also a reason they get paid the big $. However, the truth is it's never enough.

Still, this was done on the inside. That's a different level then the hacker prevention is looking for. This would be internal prevention which they should still have easily picked up on. And I agree, I want to know the excuses he used. Or how stupid his superiors are to believe them perhaps. Either way it still shows how weak security is. Regardless of where he was in the nation, if it's all u sing the same data, it should be secured just as well. The Hawaii excuse is just that. Sloppy/cheap/lazy/etc. behavior of the NSA.

 

[Nima304]

From my experiences with government computer systems, they're often pretty well-protected from the outside, and you'd need serious skill to be able to break into them remotely. However, once you're inside whatever network your target is, the security is almost hilariously nonexistent. You stop laughing once you realize that a portion of your paycheck went into building that system.

 

[amstech]

According to a "senior intelligence official", Snowden prefers Bic Macs over Whoppers.

 

[TS-56336]

Snowden did our civil liberties a favor that is a fact. Power to the people.

 

[Guest]

The difference between the NSA and corporate spying: You have to go to multiple companies to get different data such as: an isp for home (or other living area) internet activity, a cell phone company to get mobile data, ect, on a single individual and the services he/she may use. Whether it be the govt or some garage hacker. It DOES take far more time, effort, and resources this way.

Or you can just access the NSA's vast database.

"The government has spied on it's people for nearly all of history" is true.... however, that's not the issue. The issue is how easy the NSA makes it for ANYONE who is talented enough to do it. Or even just gets lucky EXACTLY like snowden did. It's like a "google" on sooo many individuals FOR anyone who has the ability to access it.

That's why the NSA is such an issue.

As for snowden, he is controversial. At the same time, he is also proof of what's going on, whether or not HOW he did was right or wrong.
Personally, I don't really see him as some hero. I also don't really know for sure he could have gone about this in some drastically different moral way either. Nothing about this is moral. Honestly, if Snowden had said he took information from the NSA with a basic web crawler without snowden releasing anything else about the NSA, would you believe that? Do you even think you would hear or read about it? How can he prove what's going on without muddying up the water so that people could actually see the mud first? The great NSA compromised by a basic web crawler and a stroke of luck?

 

[Guest]

I'm not saying he is a hero. I'm not saying he shouldn't have done what he did either. I just don't know his situation at the time. Maybe he didn't either.

 

[Guest]

"It's like a "google" on sooo many individuals FOR anyone who has the ability to access it." Snowden even brought the web crawler....

 

[chuckspires_com]

What really bothers me is the more common sense type stuff.

Our government keeps hollering and putting out warning when there is a new threat that people should take precautions to protect them self from. But yet they do not do the same and their info is much more dangerous falling into the wrong hands.

Our government is so hot to prosecute anyone whom is even suspected of the slightest probing of their networks. But yet my job is to do exactly that and I routinely find holes and vulnerabilities in that are common of unprotected networks.

Our government pretty much has an attitude of "no one would dare attack our network" but it happens everyday without fail and a good 25% of that they never ever suspect it happened till whomever did it releases the info.

Our government constantly erodes it's citizens privacy while always defending it's actions as protecting it's citizens.

Our government .... I could keep going. The facts are. Our government doesn't care because we the citizens don't make them care. Whats the saying. If the people fear it's government you have tyranny. If the government fear the people you have freedom.