Skip to content

[Flang] Add Sphinx man page and html support for Flang #141882

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 9 commits into
base: main
Choose a base branch
from

Conversation

snarang181
Copy link
Contributor

@snarang181 snarang181 commented May 29, 2025

This patch refactors the Flang documentation CMake and Sphinx configuration to address build issues.

CMake changes:

  • Moves the gen_rst_file_from_td() call out of the HTML-only block so that both docs-flang-html and docs-flang-man builds depend on the generated FlangCommandLineReference.rst file.
  • Consolidates the copy-flang-src-docs target so that both HTML and man builds use the prepared Source/ directory.
  • Ensures the docs-flang-man target explicitly depends on the gen-FlangCommandLineReference.rst generation and
    the copy-flang-src-docs directory preparation.

conf.py changes:

  • Introduces myst_parser dependency as a required Markdown parser for both HTML and man builds.
  • Introduces the correct source_suffix mapping for both .rst and .md files.
  • Populates the man_pages configuration so the main index page generates a properflang(1)man page.

Fixes #141757

Copy link

Thank you for submitting a Pull Request (PR) to the LLVM Project!

This PR will be automatically labeled and the relevant teams will be notified.

If you wish to, you can add reviewers by using the "Reviewers" section on this page.

If this is not working for you, it is probably because you do not have write permissions for the repository. In which case you can instead tag reviewers by name in a comment by using @ followed by their GitHub username.

If you have received no comments on your PR for a week, you can request a review by "ping"ing the PR by adding a comment “Ping”. The common courtesy "ping" rate is once a week. Please remember that you are asking for valuable time from other developers.

If you have further questions, they may be answered by the LLVM GitHub User Guide.

You can also ask questions in a comment on this PR, on the LLVM Discord or on the forums.

@llvmbot llvmbot added the flang Flang issues not falling into any other category label May 29, 2025
@snarang181 snarang181 changed the title [Flang][Docs] Add Sphinx man page support for Flang [Flang] Add Sphinx man page support for Flang May 29, 2025
@snarang181
Copy link
Contributor Author

@pawosm-arm -- Sorry to tag you explicitly, I do not have access to formally request reviews using the Reviewers section.

@pawosm-arm
Copy link
Contributor

@pawosm-arm -- Sorry to tag you explicitly, I do not have access to formally request reviews using the Reviewers section.

No worries, I will look into it today.

Copy link

github-actions bot commented May 29, 2025

⚠️ Python code formatter, darker found issues in your code. ⚠️

You can test this locally with the following command:
darker --check --diff -r HEAD~1...HEAD flang/docs/conf.py
View the diff from darker here.
--- conf.py	2025-06-02 20:59:23.000000 +0000
+++ conf.py	2025-06-02 21:01:52.374223 +0000
@@ -229,11 +229,13 @@
 
 # -- Options for manual page output --------------------------------------------
 
 # One entry per manual page. List of tuples
 # (source start file, name, description, authors, manual section).
-man_pages = [("index", "flang", "Flang Documentation (In Progress)", ["Flang Contributors"], 1)]
+man_pages = [
+    ("index", "flang", "Flang Documentation (In Progress)", ["Flang Contributors"], 1)
+]
 
 # If true, show URL addresses after external links.
 # man_show_urls = False
 
 

@pawosm-arm
Copy link
Contributor

I think more people need to look into it. AFAIR the plan was that flang does not use the .rst files for documentation, it was supposed to rely on the markdown files entirely.

@snarang181
Copy link
Contributor Author

I think more people need to look into it. AFAIR the plan was that flang does not use the .rst files for documentation, it was supposed to rely on the markdown files entirely.

@pawosm-arm, thanks for looking into it. I got rid of the .rst files. When I added the explicit source suffix, it seemed like the man-page build needed the parser for the .md files. With the myst_parser library not being imported, I was hitting parsing errors of the .md files, so I explicitly changed the requirement. With that and adding placeholder missing .md files, I can generate a man doc at ${BUILD_DIR}/tools/flang/docs/man/flang.1

@kiranchandramohan
Copy link
Contributor

ninja flang-doc might be generating files like FIRLangRef.md. Would adding these files manually affect that?
Note that the HTML files generated from these are displayed currently in the following pages:
https://flang.llvm.org/docs/
https://flang.llvm.org/docs/FIRLangRef.html
https://flang.llvm.org/docs/FlangCommandLineReference.html

@snarang181
Copy link
Contributor Author

ninja flang-doc might be generating files like FIRLangRef.md. Would adding these files manually affect that?

Note that the HTML files generated from these are displayed currently in the following pages:

https://flang.llvm.org/docs/

https://flang.llvm.org/docs/FIRLangRef.html

https://flang.llvm.org/docs/FlangCommandLineReference.html

It did not seem like the build process got affected locally for me. Is there any other way of testing this out?

@kiranchandramohan
Copy link
Contributor

Can you enable -DLLVM_ENABLE_SPHINX=ON -DSPHINX_WARNINGS_AS_ERRORS=OFF in the build options and try ninja docs-flang-html?
The patch that added FIRLangRef.md to docs is https://reviews.llvm.org/D128650

@snarang181
Copy link
Contributor Author

https://reviews.llvm.org/D128650

@kiranchandramohan -- this runs for me successfully and does not interfere with the placeholder file. The final FIRLangRef.md that is produced in the build artifacts has been generated by tablegen.

@kiranchandramohan
Copy link
Contributor

https://reviews.llvm.org/D128650

@kiranchandramohan -- this runs for me successfully and does not interfere with the placeholder file. The final FIRLangRef.md that is produced in the build artifacts has been generated by tablegen.

What about the man pages and its contents. Is it the empty file or the one produced by tablegen?

@snarang181
Copy link
Contributor Author

https://reviews.llvm.org/D128650

@kiranchandramohan -- this runs for me successfully and does not interfere with the placeholder file. The final FIRLangRef.md that is produced in the build artifacts has been generated by tablegen.

What about the man pages and its contents. Is it the empty file or the one produced by tablegen?

As I mentioned the man page produced at ${BUILD_DIR}/tools/flang/docs/man/flang.1 looks like the one produced by tablegen. The FIRLangRef.md looks to be completely overwritten in the build directory.
For the other placeholder file, FlangCommandLineReference.md, there is no fallout either.

@kiranchandramohan
Copy link
Contributor

https://reviews.llvm.org/D128650

@kiranchandramohan -- this runs for me successfully and does not interfere with the placeholder file. The final FIRLangRef.md that is produced in the build artifacts has been generated by tablegen.

What about the man pages and its contents. Is it the empty file or the one produced by tablegen?

As I mentioned the man page produced at ${BUILD_DIR}/tools/flang/docs/man/flang.1 looks like the one produced by tablegen. The FIRLangRef.md looks to be completely overwritten in the build directory. For the other placeholder file, FlangCommandLineReference.md, there is no fallout either.

Can we do something similar to https://reviews.llvm.org/D128650 to avoid needing dummy FIRLangRef.md and FlangCommandLineReference?

@snarang181
Copy link
Contributor Author

https://reviews.llvm.org/D128650

@kiranchandramohan -- this runs for me successfully and does not interfere with the placeholder file. The final FIRLangRef.md that is produced in the build artifacts has been generated by tablegen.

What about the man pages and its contents. Is it the empty file or the one produced by tablegen?

As I mentioned the man page produced at ${BUILD_DIR}/tools/flang/docs/man/flang.1 looks like the one produced by tablegen. The FIRLangRef.md looks to be completely overwritten in the build directory. For the other placeholder file, FlangCommandLineReference.md, there is no fallout either.

Can we do something similar to https://reviews.llvm.org/D128650 to avoid needing dummy FIRLangRef.md and FlangCommandLineReference?

@kiranchandramohan, I removed the placeholder .md files and the man pages get generated now. Just a warning of the form
WARNING: toctree contains reference to nonexisting document 'FIRLangRef'
WARNING: toctree contains reference to nonexisting document 'FlangCommandLineReference'
This is probably OK?

@tarunprabhu
Copy link
Contributor

@kiranchandramohan, I removed the placeholder .md files and the man pages get generated now. Just a warning of the form WARNING: toctree contains reference to nonexisting document 'FIRLangRef' WARNING: toctree contains reference to nonexisting document 'FlangCommandLineReference' This is probably OK?

It might not be ok. If the buildkite fails, as it does here, it will probably break buildbots too. I am not sure what is causing this though.

@snarang181
Copy link
Contributor Author

@kiranchandramohan, I removed the placeholder .md files and the man pages get generated now. Just a warning of the form WARNING: toctree contains reference to nonexisting document 'FIRLangRef' WARNING: toctree contains reference to nonexisting document 'FlangCommandLineReference' This is probably OK?

It might not be ok. If the buildkite fails, as it does here, it will probably break buildbots too. I am not sure what is causing this though.

Looking further into it.

@snarang181
Copy link
Contributor Author

@kiranchandramohan @tarunprabhu, what do you guys think about modifying index.md in the following manner ? This will load things on demand and we should not be hitting the nonexistent document errors.

```{eval}
import os
if os.path.exists('FlangCommandLineReference.rst'):
    print('FlangCommandLineReference')

@kiranchandramohan
Copy link
Contributor

@kiranchandramohan @tarunprabhu, what do you guys think about modifying index.md in the following manner ? This will load things on demand and we should not be hitting the nonexistent document errors.

```{eval}
import os
if os.path.exists('FlangCommandLineReference.rst'):
    print('FlangCommandLineReference')

I would prefer something like https://reviews.llvm.org/D128650 where the generated md/rst files are copied by the docs-flang-man target before it hits the code that triggers the nonexistent document warning/errors.

@snarang181 snarang181 force-pushed the snarang181/flang-manpage-build-fix branch from 4f24ea6 to bee779e Compare May 30, 2025 16:27
@snarang181
Copy link
Contributor Author

Hi @kiranchandramohan @tarunprabhu, with my latest commit the html target is built without any warnings. When I trigger the man build via ninja docs-flang-man,
I see

WARNING: toctree contains reference to nonexisting document 'FIRLangRef'
WARNING: toctree contains reference to nonexisting document 'FlangCommandLineReference'

If we build with -DSPHINX_WARNINGS_AS_ERRORS=OFF, this will lead to the man page successfully being built. If the flag is not set, we will hit a fatal error and fail the man build. Should we add this as a TODO and open another issue for that? I think this patch is tailored for the missing man_config issue and man not being built at all. Let me know what you guys think.

@snarang181 snarang181 force-pushed the snarang181/flang-manpage-build-fix branch from 1d69ea5 to cd886e1 Compare May 31, 2025 00:08
@snarang181 snarang181 changed the title [Flang] Add Sphinx man page support for Flang [Flang] Add Sphinx man page and html support for Flang May 31, 2025
@snarang181
Copy link
Contributor Author

@kiranchandramohan, @tarunprabhu, @pawosm-arm
Hi guys, I think this PR is ready for review. We can successfully build both HTML and man targets with these changes. Let me know how it looks.
Thanks.

@pawosm-arm
Copy link
Contributor

pawosm-arm commented May 31, 2025

Not sure what is the ultimate scope of this PR, the flang manpage is indeed being generated. There are some observations though, but I'm not sure if this is the PR which should address them. Namely:

  • At the very beginning we can see this:
FLANG |VERSION| (IN-PROGRESS) RELEASE NOTES
          warning
  • There are some mentions of the problems with building on Ubuntu 18. This is slightly old Linux distribution these days, I doubt many cares about those issues, and it doesn't feel like a manpage is a good place for running into details of them.

  • The man page is way too big (21054 lines of text and tables!), way too extensive and way too detailed. It feels like entire Flang documentation went into it. Was this intentional?

I like the fact that this page finally generates easily with no need for extensive CMake configuring.

@snarang181
Copy link
Contributor Author

  • The man page is way too big, way too extensive and way too detailed. It feels like entire Flang documentation went into it. Was this intentional?

I think the man page is intended to be that way since all the documentation is built into index.md. If we wanted more separate man pages, we can more config entries but that did not seem to be the intended use case.

@snarang181
Copy link
Contributor Author

Not sure what is the ultimate scope of this PR, the flang manpage is indeed being generated. There are some observations though, but I'm not sure if this is the PR which should address them. Namely:

I agree that there might be some more things to iron out but I think this patch has already touched quite a bit of CMake config and conf.py. It might be suitable to create another issue and address the issues in another PR as it is out of scope for the issue this is linked to.

@pawosm-arm
Copy link
Contributor

  • The man page is way too big, way too extensive and way too detailed. It feels like entire Flang documentation went into it. Was this intentional?

I think the man page is intended to be that way since all the documentation is built into index.md. If we wanted more separate man pages, we can more config entries but that did not seem to be the intended use case.

What I meant is this seems unusual. Compare these:

$ ls -la *lang*
-rw-r--r-- 1 user01 users    28437 May 31 10:26 clang.1
-rw-r--r-- 1 user01 users     1450 May 31 10:20 clang-tblgen.1
-rw-r--r-- 1 user01 users  1199536 May 31 10:26 flang.1

@snarang181
Copy link
Contributor Author

  • The man page is way too big, way too extensive and way too detailed. It feels like entire Flang documentation went into it. Was this intentional?

I think the man page is intended to be that way since all the documentation is built into index.md. If we wanted more separate man pages, we can more config entries but that did not seem to be the intended use case.

What I meant is this seems unusual. Compare these:

$ ls -la *lang*
-rw-r--r-- 1 user01 users    28437 May 31 10:26 clang.1
-rw-r--r-- 1 user01 users     1450 May 31 10:20 clang-tblgen.1
-rw-r--r-- 1 user01 users  1199536 May 31 10:26 flang.1

Ah ok, I see your point. I will defer this to someone who knows the expected size or if it really is meant to be that big.

@snarang181 snarang181 force-pushed the snarang181/flang-manpage-build-fix branch from 477e824 to ae70eed Compare June 1, 2025 23:04
Copy link
Contributor

@kiranchandramohan kiranchandramohan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the changes. This looks OK to me. I have a question inline.

LGTM. Please wait for @tarunprabhu and @pawosm-arm

add_sphinx_target(man flang)
add_sphinx_target(man flang SOURCE_DIR "${CMAKE_CURRENT_BINARY_DIR}/Source")
add_dependencies(docs-flang-man gen-FlangCommandLineReference.rst)
add_dependencies(docs-flang-man copy-flang-src-docs)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

copy-flang-src-docs seems to depend on gen-FlangCommandLineReference.rst in the definition above. So is it required to add the dependence on gen-FlangCommandLineReference.rst again?

Also, where is gen-FlangCommandLineReference.rst defined?

This patch enables building Flang man pages by:

- Adding a `man_pages` entry in flang/docs/conf.py for Sphinx man builder.
- Adding a minimal `index.rst` as the master document.
- Adding placeholder `.rst` files for FIRLangRef and FlangCommandLineReference to fix toctree references.

These changes unblock builds using `-DLLVM_BUILD_MANPAGES=ON` and allow `ninja docs-flang-man` to generate `flang.1`.

Fixes llvm#141757
Due to that, the myst_parser was explictly imported.
Adding Placeholder .md files which are required by index.md
copy-flang-src-docs builds Source/ directory
and in fixing, did not include it in the HTML target deps
@snarang181 snarang181 force-pushed the snarang181/flang-manpage-build-fix branch from ae70eed to b94047e Compare June 2, 2025 10:26
@snarang181
Copy link
Contributor Author

Thanks for the changes. This looks OK to me. I have a question inline.

LGTM. Please wait for @tarunprabhu and @pawosm-arm

Thanks for the review, @kiranchandramohan. Whenever @tarunprabhu and @pawosm-arm, I request someone with commit access to merge the PR, I do not have the access.
Thanks again.

@pawosm-arm
Copy link
Contributor

pawosm-arm commented Jun 2, 2025

I'm in two minds here. I've tried this PR after forced push, yet nothing really has changed from my previous comment (there's still this warning word visible, and there's still unresolved |VERSION| token). But this isn't the main point here. The main point is that whatever man flang shows has a content indeed, quite extensive one, but it is not a manpage. It doesn't have the manpage structure. There are no SYNOPSIS, DESCRIPTION, OPTIONS or any other sections typical to a real manual page. Maybe at this step we should only create an empty stub (with only the NAME section) instead? Then in a followup PR it could be filled in with a contents similar to what man clang shows?

The true benefit of this commit is that the building process continues even with -DFLANG_INCLUDE_DOCS=ON passed to CMake. But the cost of it is that we're now getting a page which for some may play a role of the flang manpage stopping others from writing a real one.

@snarang181
Copy link
Contributor Author

I'm in two minds here. I've tried this PR after forced push, yet nothing really has changed from my previous comment (there's still this warning word visible, and.

By the warning, you mean inside the contents of the man page right? Not the build process giving warning to nonexistent documents.

@pawosm-arm
Copy link
Contributor

Something to consider: maybe we should introduce the use of something like this? https://github.com/cpuguy83/go-md2man

@pawosm-arm
Copy link
Contributor

I'm in two minds here. I've tried this PR after forced push, yet nothing really has changed from my previous comment (there's still this warning word visible, and.

By the warning, you mean inside the contents of the man page right? Not the build process giving warning to nonexistent documents.

Yes, as I quoted it above:

FLANG(1)                               Flang                              FLANG(1)

NAME
       flang - Flang Documentation

       Flang  is  LLVM's  Fortran frontend that can be found here. It is often reâ
       ferred to as "LLVM Flang" to differentiate itself from  "Classic  Flang"  -
       these are two separate and independent Fortran compilers. LLVM Flang is unâ
       der active development. While it is capable of generating executables for a
       number  of  examples,  some functionality is still missing. See Getting Inâ
       volved for tips on how to get in touch with us and to learn more about  the
       current status.

FLANG |VERSION| (IN-PROGRESS) RELEASE NOTES
          warning

          These  are  in-progress  notes  for the upcoming LLVM |version| release.
          Release notes for previous releases can be found on the Download Page.

@snarang181
Copy link
Contributor Author

I'm in two minds here. I've tried this PR after forced push, yet nothing really has changed from my previous comment (there's still this warning word visible, and.

By the warning, you mean inside the contents of the man page right? Not the build process giving warning to nonexistent documents.

Yes, as I quoted it above:


FLANG(1)                               Flang                              FLANG(1)



NAME

       flang - Flang Documentation



       Flang  is  LLVM's  Fortran frontend that can be found here. It is often reâ

       ferred to as "LLVM Flang" to differentiate itself from  "Classic  Flang"  -

       these are two separate and independent Fortran compilers. LLVM Flang is unâ

       der active development. While it is capable of generating executables for a

       number  of  examples,  some functionality is still missing. See Getting Inâ

       volved for tips on how to get in touch with us and to learn more about  the

       current status.



FLANG |VERSION| (IN-PROGRESS) RELEASE NOTES

          warning



          These  are  in-progress  notes  for the upcoming LLVM |version| release.

          Release notes for previous releases can be found on the Download Page.

Ok, I see your point. I think a follow up PR to iron out the contents of the man page should be the course of action since the issue this fix links to is for the build process only. The changes might get too convoluted; what do you think?

@pawosm-arm
Copy link
Contributor

Something to consider: maybe we should introduce the use of something like this? https://github.com/cpuguy83/go-md2man

This seems somewhat simpler: https://kristaps.bsd.lv/lowdown

@pawosm-arm
Copy link
Contributor

Something to consider: maybe we should introduce the use of something like this? https://github.com/cpuguy83/go-md2man

This seems somewhat simpler: https://kristaps.bsd.lv/lowdown

From the other side, the manpage flang-doc generates from markdown demonstrates that we are able to make markdown to manpage conversion without engaging any external tools. What we lack is an input which could establish the correct structure, something that clang has (see https://github.com/llvm/llvm-project/blob/main/clang/docs/CommandGuide/clang.rst) but in the markdown format. I'd rather see this PR introducing a stub for such content rather than generating a substantial in size content made of something which isn't a manpage.

@snarang181
Copy link
Contributor Author

Something to consider: maybe we should introduce the use of something like this? https://github.com/cpuguy83/go-md2man

This seems somewhat simpler: https://kristaps.bsd.lv/lowdown

From the other side, the manpage flang-doc generates from markdown demonstrates that we are able to make markdown to manpage conversion without engaging any external tools. What we lack is an input which could establish the correct structure, something that clang has (see https://github.com/llvm/llvm-project/blob/main/clang/docs/CommandGuide/clang.rst) but in the markdown format. I'd rather see this PR introducing a stub for such content rather than generating a substantial in size content made of something which isn't a manpage.

Sounds good.
Can we agree on a stub for the purposes of this PR?
Cc @kiranchandramohan @tarunprabhu

@tarunprabhu
Copy link
Contributor

I am not very familiar with the way the manpage for clang is generated. Does clang.rst have to be manually kept up to date with any command line options that are added?

As far as a stub goes, does it make sense to write a barebones .rst such as clang's, but in the contents just provide a link to flang's online command line reference? We could add text that the man page is a work in progress and that the link to the online documentation is temporary.

@snarang181
Copy link
Contributor Author

As far as a stub goes, does it make sense to write a barebones .rst such as clang's, but in the contents just provide a link to flang's online command line reference? We could add text that the man page is a work in progress and that the link to the online documentation is temporary.

To be clear, the generated flang man page should have something like

Flang Manual Page (In Progreess) 

This man page is under development. 
For full documentation, please visit [Flang's Online Command Line Reference](https://flang.llvm.org/docs/FlangCommandLineReference.html)

Does this proposal have consensus?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
flang Flang issues not falling into any other category
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Flang] man pages cannot be built
5 participants