WebAIM - Web Accessibility In Mind

E-mail List Archives

Re: I need a List auf all sites auf a Website

for

From: Chase, DJ
Date: Jan 16, 2024 7:35AM

Hi Harry,

You should know that no crawler/indexer can always find 100% of the pages on all sites, so your best bet is to first see if the site you're testing has a sitemap. That will list every page they want indexed in search engines, so will give you a very large sample of the pages on the site. It probably won't include pages like the user settings or other pages that require you to log in, but those should be pretty easy to find on their own as well. (With very few exceptions, crawlers/indexers will also not find those pages on any site as well).

If you can't find a sitemap, it might also be worth briefly checking /robots.txt (https://your-domain.example/robots.txt). That lists what pages or folders sites explicitly do or do not want indexed, so some sites just use that as their sitemap.

Cheers,
--
DJ Chase
Digital Accessibility Specialist, NECC
They, Them, Theirs
+1 978 556 3794

From: WebAIM-Forum < <EMAIL REMOVED> > on behalf of Ron Zuidema < <EMAIL REMOVED> >
Sent: Sunday, January 14, 2024 03:32
To: <EMAIL REMOVED>
Cc: <EMAIL REMOVED>
Subject: Re: [WebAIM] I need a List auf all sites auf a Website

[You don't often get email from <EMAIL REMOVED> . Learn why this is important at https://aka.ms/LearnAboutSenderIdentification ]

ATTENTION: This email is from an external source. Do Not click on any links or open any attachments unless you recognize the sender and know the content is safe. To report a suspicious email, please use the new phishing alert button in outlook

Hi Harry,

Before conducting a WCAG audit, I find it helpful to use Screaming Frog to gain insight into a website.

Screaming Frog crawls a website and provides various information, such as the HTML pages and structures. This tool may also be beneficial for you, although there may be a slight learning curve. However, it is not difficult to use for general purposes like this.

You can learn more about it here: https://www.screamingfrog.co.uk/seo-spider/ (I am not affiliated).

Best, Ron

From: <EMAIL REMOVED> < <EMAIL REMOVED> >
Sent: Saturday, January 13, 2024 18:50
To: 'WebAIM Discussion List' < <EMAIL REMOVED> >
Subject: [WebAIM] I need a List auf all sites auf a Website

Hi all,

Is there a software or a website that lists all pages and subpages of a website?

Thanks for your suggestions

kind regards

Harry