Doc bundle size

[vogella]

I recently removed the ICU bundle from the Eclipse Platform, reducing the Eclipse SDK download by ~15MB.
While updating a client IDE in a slow environment (v-client) I noticed the docs bundles are huge.

JAR sizes of the shipped doc bundles:

Bundle	JAR size
org.eclipse.platform.doc.isv	42M
org.eclipse.jdt.doc.user	9.1M
org.eclipse.jdt.doc.isv	7.9M
org.eclipse.pde.doc.user	6.0M
org.eclipse.platform.doc.user	3.8M

The working trees are far larger (600M+ for platform.doc.isv) but git-ignored.

For example, org.eclipse.platform.doc.isv:
the JAR is 42M compressed (~254M uncompressed) and is almost all generated Javadoc.

HTML category	Uncompressed	Files	Share
Generated Javadoc (reference/api/)	230.2M	11,049	98.4%
Other HTML	3.8M	696	1.6%

So ~90% of the entire uncompressed JAR is generated Javadoc.
The same concentration applies to org.eclipse.jdt.doc.isv.

How is API info used to my knowledge:

• AFAIK PDE uses the bundled Javadoc for hover/F1 only when the matching source bundle is not attached. This is offline; AFAIK if online it could point to online location. The full SDK ships source bundles, so editor hover/F1 renders Javadoc from these bundles. For that setup the bundled HTML is redundant.
• The same reference is online at help.eclipse.org
• Maybe the generated API is used for the API baseline comparison (not sure about this)

Possible directions:

• Ship the API Javadoc as a separate, optional feature instead of bundling it by default.
• Point org.eclipse.pde.core.javadoc at the online reference instead of a bundle if that is possible

I do not plan to work on this myself, but AFAIK @HannesWell is actively optimizing the build and download size, so this might be of interest to him.

[iloveeclipse]

Maybe we can keep org.eclipse.platform.doc.isv & org.eclipse.jdt.doc.isv in the SDK p2 repository but not add them to the SDK binary. So webhelp (or any other product) can include the documentation bundles from p2 update site and SDK itself doesn't need to include them? I personally never use API Javadocs from inside Eclipse, but google search time to time points to some reference pages.

[HannesWell]

I assume the rational in the past was that the SDK should contain all developer resources, which of course includes most artifacts, their sources and the full doc/help. Personally I also don't use it within Eclipse and if necessary, usually in the browser.
I wouldn't be surprised if most users of the SDK do the same.

Maybe we can keep org.eclipse.platform.doc.isv & org.eclipse.jdt.doc.isv in the SDK p2 repository but not add them to the SDK binary. So webhelp (or any other product) can include the documentation bundles from p2 update site and SDK itself doesn't need to include them?

Yes that would be possible and shouldn't be difficult.

But the SDK products are also used in I-build tests, so I wonder if some tests depend on the doc bundles being present? Does anyone of you know that?
If any doc-bundle is necessary, it maybe can be installed from the p2 repo. I'd have to check that.

[vogella]

I think we have TocLinkChecker in the aggregator.

[iloveeclipse]

But the SDK products are also used in I-build tests, so I wonder if some tests depend on the doc bundles being present?

Doing plain text search for org.eclipse.platform.doc.isv I don't see obvious candidates.
org.eclipse.jdt.doc.isv seem to be used by org.eclipse.ua.tests.help.AllHelpPerformanceTests but these tests are not executed.

[HannesWell]

I think we have TocLinkChecker in the aggregator.

Yes that test indeed require all doc bundles, which is also reflected in the project's pom.
https://github.com/eclipse-platform/eclipse.platform/blob/dbb288406068e9249f20f1f1eba0b958cb81e199/ua/org.eclipse.ua.tests.doc/pom.xml#L112-L136

And org.eclipse.ua.tests.doc is executed in I-builds. So the doc bundles would have to be installed into the test runtime.
Since each I-build test has a P2 director provision step that should generally be possible but it still has to be tested if it also works in that particular case.

But I'll probably won't have time for that in June.
But since this is not urgent (IMO) and I plan to work on streamlining the test setup in July and assume that testing this would then become simpler, I suggest to investigate that together.

[laeubi]

So are we really talking about the SDK.zip here or that Eclipse ships with the docs bundles? This reads like the later:

While updating a client IDE in a slow environment (v-client) I noticed the docs bundles are huge

As far as I know these are part of the offline help of Eclipse so if one wants to remove that it would likely be changed to allow shipping Eclipse products only with the online help.