Removing data from text file except specified URL (data scraping help)

By H3llion
Apr 16, 2014
Post New Reply
  1. Hi guys, I have a slight problem after my first data scraping session.

    Basically I have around 500k of lines and around 50k "" urls that are in this text file. However the only part I need is the url and everything else is redundant. The URL is dynamic so it's different username everytime as well as on a random line rather then continuous every 5 lines or so.

    Is there a software out there or maybe a notepad++/sublime2 algorythim that takes out all the data or simply extracts the data I want?

    Here is what it looks like:

    Code:,,,Landon Austin - Armor - Official Music Video Download on iTunes!! - <a href="" target="_blank" title="" rel="nofollow" dir="ltr" class="yt-uix-redirect-link"></a> Download on Amazon!,,,,,Download on iTunes ♪ For more information ☞ smtown hompage : <a href="" target="_blank" title="" rel="nofollow" dir="ltr" class="yt-uix-redirect-link"></a> ☞ smtown EXO-KÂ*...,,,,,Watch the official music video by Kerbera for their hit single &quot;Counterpoints For more information on Kerbera check them out onÂ*...,,,,,Just a little video I put together cuz I was bored as hell. Trying to brush up my skills. Video is scenes from the movie 300 and musicÂ*...,,,,,*** it All (Honest Final Exam Version) Music Video Like the video=) Facebook:<a href="" target="_blank" title="" rel="nofollow" dir="ltr" class="yt-uix-redirect-link"></a>Â*...,,,,,Music video for the song &quot;Pink Print&quot; by Antillectual taken from the album &quot;Perspectives & Objectives&quot;. Order the album atÂ*...,,,,,From &quot;International&quot; available June 10th 2014 on Sacred Bones Records Directed by Cali Thornhill Dewitt Shot and Edited byÂ*...,,

    I am using a Youtube scraper to get description from specific videos (music covers) and the descriptions have urls that I require (for instance, let say, Soundcloud). Now the Scrapper pulls in the WHOLE description, however all I need is the ''Soundcloud'' link.
    Last edited: Apr 16, 2014

Similar Topics

Add your comment to this article

You need to be a member to leave a comment. Join thousands of tech enthusiasts and participate.
TechSpot Account You may also...