E-mail List Archives
Thread: CSS spiders
Number of posts in this thread: 2 (In chronological order)
From: Cliff Tyllick
Date: Tue, Jul 09 2013 6:24PM
Subject: CSS spiders
No previous message | Next message →
Is there a tool—or perhaps a way to use a common tool like Google Analytics or Webtrends—to spider a site, identify what stylesheets are in use, and label them according to directory containing pages that call them?
For example:
Stylesheets called by all pages in or below /top-level-directory-1:
relative-path/css-1.txt
/top-level-directory-2/path/css-2.txt
Stylesheets called by all pages in or below /top-level-directory-2:
relative-path/css-2.txt
relative-path/css-2b.txt
/top-level-directory-1/path/css-1.txt
/top-level-directory-3/path/css-3.txt
And so on.
Seems to me the solution might be trivial. Even so, if it already exists, using what exists would be easier than trying to re-create it.
Thanks!
Cliff Tyllick
@clifftyll
From: Steve Green
Date: Tue, Jul 09 2013 7:05PM
Subject: Re: CSS spiders
← Previous message | No next message
You could do this with Xenu's Link Sleuth - http://home.snafu.de/tilman/xenulink.html - although it won't give you the results in the format you want so you would have to do a bit of work.
Xwnu is a website mapping tool that we use on virtually every project. Whilst the concept of mapping or spidering sounds simple there can often be a lot more to it than you would imagine. You often need to add subdomains and http/https variants or exclude certain parts of the website such as forums because they are too large. Occasionally you get badly-built websites that contain recursive URLs so the website appears to have an infinite number of pages.
After mapping the website you will have a list of all the files Xenu has found. Sorting by the MIME type will group the CSS files together. You can then view the Properties for each CSS file in turn, and Xenu will show you which pages link to them. You can copy these lists of pages into a spreadsheet where they can be manipulated more easily.
If anyone has any difficulty using Xenu please contact me off-list and I would be happy to help.
Steve Green
Test Partners Ltd