Search Console 404 despite good deploy

Hi,

I’m deploying, including manually deploying, a static site built with docusaurus. It’s all fine as far as I can see - the pages render well on multiple devices and very fast. However…

I’m having a hard time to get Google to even believe the pages are there. I’ve tried URL submission over Search Console (comes back 404), I’ve tried pagespeed insights (comes back 404). Tried bing URL inspection - same thing. I’ve also tried that ol’ fashioned way of just waiting to see what happens 72 hours later - still 404ing.

Now, interestingly, when I pull up a private browser, I get a 404 for a fraction of a second, then it redirects to the real page. Only happens occasionally and is too quick to catch what’s happening. Have recreated this on various laptops, mobiles, VPNs and same thing.

Here’s a link to check (sorry, work in progress, ignore the content): https://nakedglue.com/Glue-this-to-that/Glue-Ceramic-to

Anyone else see this?

I was able to observe the same thing - the page loads and gives me a 404 and then directs to the page.

curl shows the 404 as well:

$ curl -I https://nakedglue.com/Glue-this-to-that/Glue-Ceramic-toHTTP/2 404date: Tue, 14 Jun 2022 10:15:40 GMTcontent-type: text/plain; charset=utf-8cf-ray: 71b24e5c29d3407e-LHRvary: Accept-Encodingcf-cache-status: DYNAMICexpect-ct: max-age=604800, report-uri="https://report-uri.cloudflare.com/cdn-cgi/beacon/expect-ct"x-content-type-options: nosniffserver: cloudflarealt-svc: h3=":443"; ma=86400, h3-29=":443"; ma=86400

which will explain why the bots are getting a 404.

Loading the page with a page inspector open, the HTTP response code is a 404 but then the content is loaded in - perhaps with javascript?

John B

Yea, this is super-weird. On inspection with dev tools the 404 was there but then the rest of the page loaded fine, but naturally the bots were freaking out on that first 404.

Did wonder whether it was on the hosting side (I use render.com), so tried it with Netlify and Vercel. Same thing happened, and again just with that one page.

I found a work-around (not nice one) which was to create a new doc page with different URL, then copy the markdown across. I’m autogenerating everything in sidebars config and it’s not referred to in the main config, so no other changes made - just the URL.

It worked fine, no 404, now being indexed. Hmmm…

Happy to share the commit ID that was broken if anyone wants to look further into this, but need to get on this side so happy that the hack-ish solution worked.

Out of curiosity, I created a docusaurus site using npx create-docusaurus@latest my-website classic and deployed it to us and I don’t see the same behaviour (https://jb-render-docusaurus.onrender.com/) so it feels like it’s something in your project here especially if it occurs on Netlify and Vercel for the same repo.

John B