Switch from directory to file format in Astro

[krzysdz]

The "file" format generates name.html files instead of default name/index.html. This would make old links work, even without redirects on Cloudflare level, while still allowing the current link format with no extension. The only observable difference between current version and this PR is that the URLs no longer end with a / (GitHub/Netlify serve a 301).

References:

• https://docs.astro.build/en/reference/configuration-reference/#buildformat
• Old URLs ending in ".html" are 404s; should they redirect instead? #2332
• @jonchurch's suggestion on Slack

[netlify]

✅ Deploy Preview for expressjscom-preview ready!

Name	Link
🔨 Latest commit	dfb0206
🔍 Latest deploy log	https://app.netlify.com/projects/expressjscom-preview/deploys/6a31da259d80c50008fb3e55
😎 Deploy Preview	https://deploy-preview-2356--expressjscom-preview.netlify.app
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.
Lighthouse	1 paths audited Performance: 100 (🟢 up 3 from production) Accessibility: 100 (no change from production) Best Practices: 100 (no change from production) SEO: 100 (no change from production) PWA: 80 (no change from production) View the detailed breakdown and full score reports
🤖 Make changes	Run an agent on this branch

---

To edit notification comments on pull requests, go to your Netlify project configuration.

[krzysdz]

EDIT: Fixed by #2390

Original comment about /index in search results

This breaks search results for API pages, which for some reason end with /index. I'm not sure if this will fix itself (after sync) or if slightly changing source file structure is necessary (krzysdz/expressjs.com@64ecc18 krzysdz/expressjs.com@981a172). I think that the current search behaviour is unexpected and should be changed whether this PR is merged or not.

@bjohansebas is there any reason why API docs live as index.mdx in separate directories instead of just name.mdx like other pages?

EDIT: Either API docs files must be moved and renamed or the code responsible for uploading pages to Orama must be changed to strip /index:

expressjs.com/scripts/orama-documents.mjs

Lines 52 to 72 in ec1d037

	export const getDocs = async (lang) => {
	const baseDir = join(CONTENT_DIR, `docs/${lang}/5x`);
	const mdFiles = await collectMdFiles(baseDir);
	return Promise.all(
	mdFiles.map(async (file) => {
	const fullPath = join(baseDir, file);
	const raw = await readFile(fullPath, 'utf-8');
	const { data, content } = matter(raw);
	const pathSegment = relative(baseDir, fullPath).replace(/\.mdx?$/, '');
	return {
	title: data.title ?? basename(file).replace(/\.mdx?$/, ''),
	description: data.description ?? '',
	content: mdToText(content),
	path: `/${lang}/${pathSegment}`,
	category: 'menu.main.docs',
	};
	})
	);
	};

What should work instead of renaming files (krzysdz/expressjs.com@f1151f6 krzysdz/expressjs.com@4b1e61b):

diff --git a/scripts/orama-documents.mjs b/scripts/orama-documents.mjs
index 0b9a9668..5dd9831a 100644
--- a/scripts/orama-documents.mjs
+++ b/scripts/orama-documents.mjs
@@ -106,7 +106,7 @@ export const getApi = async (lang) => {
           const fullPath = join(baseDir, file);
           const raw = await readFile(fullPath, 'utf-8');
           const { data, content } = matter(raw);
-          const pathSegment = relative(baseDir, fullPath).replace(/\.mdx?$/, '');
+          const pathSegment = relative(baseDir, fullPath).replace(/(\/index)?\.mdx?$/, '');

           return {
             title: data.title ?? basename(file).replace(/\.mdx?$/, ''),

EDIT 2: This applies also to /resources/index.mdx and /resources/middleware/index.mdx, so doing it while uploading docs to Orama seems like a safer solution.

[krzysdz]

As a bonus, with the changes from this PR we can just enable --include-fragments and the results won't be full of false positives.

In fact using lychee with --include-fragments on this PR is how I found #2363 and #2387.

[bjohansebas]

Okay, quite a bit of time has passed. Other than the .html extensions, have we run into any additional issues?

To be honest, I feel like we should move forward with the current approach, especially considering the redirects that have already been put in place (which I don't have the ability to remove anyway).

[krzysdz]

Okay, quite a bit of time has passed. Other than the .html extensions, have we run into any additional issues?

I don't think so. The thing that I found after creating this PR is that it just works with --include-fragments in lychee, which I think that we should enable. Fragment checking should also work with the current directory based version with more remapping.

To be honest, I feel like we should move forward with the current approach, especially considering the redirects that have already been put in place (which I don't have the ability to remove anyway).

The file format works with and without redirects (/page and /page.html just work, /page/ receives a 301 redirect to /page).

I initially wrote that this also lets us avoid 301s that add /, but while most links in menu don't have a trailing / (/api/express/ is the only one ending with /), links in content now have / appended since #2193 was merged (2 days ago).