Posts

Showing posts from August, 2013

Recursively Find Hyperlinks In A Website

I was trying to write a script to crawl a website and fetch all the hyper links pointing to all the a particular file type e.g. .pdf or .mp3. Somehow the following command did not work for me.
wget -r -A .pdf <URL>
It did not go recursively and download all PDF files. I may have to ask in  stackoverflow.

Anyway I wrote my script in python and it worked well. At least for the site I was trying crawl. The following scripts give all the absolute URLs pointing to the desired type of files in the whole website. You may have to add few more strings in excludeList configuration variable to suite your target site else you have end up infinite loop.

[code language="python"]
import re
import urllib2
import urllib

## Configurations
# The starting point
baseURL = <home page url>
maxLinks = 1000
excludeList = ["None","/","./","#top"]
fileType = ".pdf"
outFile = "links.txt"

#Gloab list of links already visited , don't want to ge…

Getting started with XBMC

Image
XBMC is a free and open source software media player for various OS platforms especially mobile. This is very useful to convert your TV dongles e.g. Android PC or Apple TV or Raspberry PI to a media center in your TV. This can not only organize and play your local media but also can stream movies and TV series. I downloaded the Android APK and installed my RocketChip MK806 Android TV.



To begin with I scanned all my mp3 and videos in my SD card using XBMC and all of them were ready to play. I found couple of plugins which listed almost all recent movies and TV series very well organized by season and episodes. As of now I have installed

Mash Up ( Installation Steps )
1 Channel ( Installation Steps )



Making it Full screen : However you can notice that the android navigation bar at the bottom always appears (Even during movie play). This is sometimes distracting. So I found an app Fullscreen which can help you get rid of this navigation bar.

Start on Boot : Also If XBMC is the only app you ar…

Moving the blog again

A year ago I moved from free shared Linux hosting to paid one to GoDaddy. Everything was good, there was no downtime like free hosting solutions. WHen I was in free hosting sometimes my site got blocked by antivirus software because somebody else would have hosted such content in the same server.  However I did not earn any revenue from this blog so paying for hosting was not really my favorite idea.

I read several articles in the internet why GoDaddy is not a very good choice for hosting blogs. I had also used GoDaddy for hosting the website of local chapter of IEEE section (ieeehyd.org) but faced a lot of problem during renewal. First of all only the 1st year hosting price was attractive but the renewal charge was almost four times than that I initially paid for starting the hosting. I had my credit card in GoDaddy payment methods and now they wouldn't let me remove it until I gave details of another card. This inspired me to close the account itself. Also lately my site got bloc…