I am seeing submit-url.php cause the browser to load crawler.php twice when inspecting in the chrome browser. One submit has the post header with the url submitted but the other is blank without the url passed as a post variable. This causes the crawler script not to run and the submit-url to return with "Re enter your url' even though a valid url was entered and it is displaying correct url in green. I am not very good with javascript so, I am not sure the right way to fix it. There are lots of posts on stackoverflow about jquery .ajax resulting in double submits of forms.

I should clarify this error can be intermittent. I think it may have to do with how fast the server is processing the double submit and which order they end up completing. Maybe whether the one with the post header of url entered is first or second. It is very strange. It just goes from working to not working then back.

Your script is really good, both the design and the idea. Just tried but the URL submission option didn't work. No database table for sites are being created when you submit a new url.