Assets folders that return a 404 error

So we start a website, add our first page, index.php or index.html, and within the first few minutes we need some images, so we create a folder called images or assets and shove a whole bunch of images inside it, generally thats the end of the story, however what if we navigated to that page?

www.mysite.com/images/
Well we would probably see a directory listing provided by the Apache web server which is obviously a pretty large security flaw, so we add an entry into our .htaccess file like Options -Indexes and now people are not going to be able to look at our directory structure. Great, so is that the end of the story?

Well what if we navigate to
www.mysite.com/images/ again, we are going to be faced with a default or custom 404 error page.

So my question is, what are other people doing with this, do we redirect the person to our home page, do we add an index file into that directory so people do not get an error, or do you like me just ignore it and never really think about it again.
Would this be seen from an SEO standpoint as a website issue, I mean a site should not have 404 errors at all.

Please let me know what you guys do, if anything, and possibly give a reason why, because honestly in a decade, i have never really thought of it till today, i’m pretty sure Mr @Dave must have some thought on this, well at least i hope he does, or maybe I am the odd one out who has never thought about the repercussions if any of this behaviour.

Hello Paul,

Do you want to allow users to browse this folder or not? Usually you don’t want to, so you can disable directory listing in htaccess, which will return 403 (forbidden) and not a 404 (not found) message.

Well, how would the search engine know about this URL and reach it? Have you linked this url to some href like <a href="www.mysite.com/images/" ? If not - the search engine won’t just randomly try to access some folder on some server :slight_smile:

Thanks Teo, Im not sure why mine is giving a 404 error, and not a 403, even though i turned off directory listing with the .htaccess -Indexes command.

No, not particularly, was just wondering if the SERPs crawler would crawl the folder considering it is part of every img src tag all over the website, even though I have added a record into my robots.txt file telling the SERPs not to crawl that folder.

Also if I would like my images entered into the google images listing, should I maybe not be restricting the robots.txt on this folder?

The only time i may have a situation like this would be in situations where I have a downloads folder then I may have an href tag like http://www.mysite.com/downloads/file.pdf however it would never be just to the actual base folder.

From the sounds of what you saying though it seems like I can safely ignore this, as no link is ever placed to just the source folder, so the crawlers can use it as part of a path to the child files and folders quite safely even if the parent folder has no index file itself?

Just by the way, the reason any of this even entered my head is because of that silly WordPress, if a browse to http://www.mysite.com/wp-content/ on a brand new WordPress install it just shows me a blank page, but does not give any error at all, and the URL does not change, and it got me wondering.

That’s right.

1 Like

Thank you, thats what i needed to hear, in 2030 I may think about this again, haha

1 Like

Could drop a .htaccess file in to the assets/images folder similar to the following:


SetEnvIfNoCase Referer example.com internal
SetEnvIfNoCase origin example.com internal

order Deny,allow
Deny from all
allow from env=internal


That would deny/restrict access to the directory to only your specified origin domain (replacing example.com of course, and denying ‘hot linking’). Also for important one time only downloads you could destroy the folder/file upon the user downloading the file, and password protect a .zip containing said file as a secondary measure.

:slight_smile:

1 Like

I knew you were nutty enough to have thought about this before me, it only took me a decade of web dev before i even wondered, lol.

Thanks Dave, makes sense.

2 Likes

Hahahaha made me smile!

1 Like