Demand

Problem Description And Context

Over the last 20 years, electron microscopy has developed from a technique that provides magnified images to become a complex ultra-high resolution analytical tool. Data sets are no longer single qualitative images, but collections of quantitative multi-channel signals. Image forming conditions are changed in a controlled manner and dynamic experiments are performed within the microscope while recording a stream of data that is then analysed to extract information about the sample.

This new approach to using an electron microscope makes developing software to control the experiments and to process the resulting data an integral part of creating and improving electron microscopy techniques. Developers in this area need a suitable software platform to build their innovative solutions on, whilst research in this area yields new software, hardware and formats for data.

Currently, the software landscape for electron microscopy is split into three categories: Closed Source software that is developed by hardware vendors to control their own products; Closed Source commercial application software for data acquisition and analysis; and data analysis and visualization routines that scientific research groups develop as part of their activities and share under varying terms and conditions.

Recently, pixelated detectors have been developed for scanning transmission electron microscopy (STEM) with resolutions of 256 × 256 pixels and better, frame rates above 1000 frames per second, and exceptional dynamic range and radiation hardness. For the first time, such detectors allow efficient capture of a complete frame of the scattered beam intensity for each scanned pixel. Such novel pixelated detectors have been used to measure nanoscale electromagnetic fields, ptychography for mapping the structural properties of materials at both nm resolution and atomic resolution. However, software support for these detectors is still in its infancy, with image capture and very basic analysis capabilities provided by the detector vendors, and one-off experimental data analysis scripts developed by individual users, including our own institute. The goal of the VIDEO project is to combine these efforts and create full software support for pixelated STEM as a starting point for a fully open, interoperable and standardized software platform for electron microscopy.

Current Solutions

The current software landscape makes it very difficult to create interoperable solutions, both for vendors and for users. Vendors are currently reluctant to open the interfaces to their products because they see this as a method to keep competitors out. Code written by scientists for individual research projects does not always meet the quality and professional support requirements necessary for production-quality systems and may be released under incompatible licenses.

Automated workflows for electron microscopy are particularly difficult to implement under such conditions. Users are currently required to repeatedly perform routine tasks manually, simply because a vendor only provides a graphical user interface to control their product and not an application programming interface (API). For that reason, standardized interfaces and a robust shared software infrastructure on which to build innovative solutions such as, for example, active feedback to stabilize instrumental conditions must be established in electron microscopy.

Furthermore, Open Science requires that the data and metadata generated in experiments can be accessed and re-analysed in a reproducible manner. To achieve this goal, data formats need to be standardized and open, and the processing algorithms must be documented and accessible, ideally in the form of Open Source so that the processing steps between raw data and result can be reproduced and analysed in appropriate detail. The undocumented file formats and Closed Source nature of current software for electron microscopy are a major obstacle to the advancement of electron microscopy and its practical applications.

Benefits Of New Solution

Electron microscopy is one of the fundamental analytical tools for science and technology in both industry and academia, with tens of thousands of instruments around the globe. Almost any transmission electron microscope can be upgraded with the detectors and software for pixelated STEM, thereby greatly enhancing the capabilities and useful life of older instruments. A strong platform for software development is particularly important for the success of pixelated STEM because software is essentially taking over the function of electron lenses; innovation will therefore be driven by improvements of the software.

Open Standards and a collaboratively developed software platform for instrument control, data acquisition and data analysis will make it easier for hardware vendors to integrate and support their products on existing electron microscopes. It will enable a number of additional business models such as the development of customised microscopy and automation solutions. Furthermore, it will empower end users to participate and control the development of the tools that they use, and it will create the conditions for users to share their methods and data, and to contribute improvements back to the project.

The open infrastructure that will be created with this PoC project is expected to expand its scope step by step towards general electron microscopy after it has established itself for pixelated STEM. At every step, the project will help to make software and method development more efficient by pooling resources for a shared infrastructure instead of re-inventing the wheel at each user's and manufacturer's location. It will allow innovative solutions such as active feedback for stabilization to be developed that combine automated control of different aspects of an electron microscopy and, ultimately, give users improved tools at lower cost to perform their work in research and development.

Open Source and collaborative software development can have many advantages when compared to Closed Source models (Table 1). As a drawback, they usually cannot rely on revenues from software sales to finance their expenses. They have to encourage stakeholders to support the project with money or in-kind contributions instead while receiving the final product at no charge. Each contribution should therefore have immediate benefits for the contributing party, such as recognition, support or influence.

The greatest driver for any successful Open Source project is the demand from users who require such software for their daily work. Practical, useful software will attract new users who will then extend the software for their own purposes and re-contribute these changes to the project. Combining all the separate efforts by different groups, including ours, into an attractive minimum viable product (MVP) is therefore a core activity within the project. This "lean startup" approach will validate that the project achieves sufficient market traction to drive standardization efforts for APIs and file formats, and build sufficient competitive pressure for hesitant vendors to open the interfaces to their products.

Table 1: Overview of immediate benefits of collaborative Open Source projects

Feature	Stakeholder	Benefits
Open Source	Scientific users	++Full control over experiments.
		++Unrestricted modification and publication of code and data.
		++Free redistribution of novel methods for maximum impact.
		++Partners with less funding are able to participate.
	Commercial users	++Free to optimize and extend for own use and integration.
	Hardware and software vendors	++Lower entry barrier and shorter time-to-market.
		++Lower development and support cost for application software.
		++Building on existing code with a focus on improvements.
	Funding agencies	++Lower license costs for funded users.
		++Funded work is freely available, thereby maximizing impact.
Collaborative development	All stakeholders	++Developments are tailored to the needs of stakeholders.
		++Diverse partners bring a broad portfolio of expertise.
		++Development, support and maintenance on many shoulders.
		++Pooling of resources instead of re-inventing the wheel.