Enhancing Global Research Impact and Saving Lives
How the containerization of climate solutions is enhancing global research impact at Karlsruhe Institute of Technology
Key metrics
Business overview
Business Focus
German research institute focused on climate and environmental sciencesBased in Europe
Key Need For Portainer
To simplify container management for scientists and researchers who may not have extensive technical expertise.Container Platform
Docker SwarmIntroduction
The Karlsruhe Institute of Technology (KIT) is a prominent German research institute focused on climate and environmental sciences. KIT is part of the larger Helmholtz Association, which encompasses some of the biggest environmental research centers in Germany. The institute is at the forefront of a significant shift in academic research towards more product-oriented outcomes, developing tools like dashboards, decision support systems, and web portals alongside traditional scientific papers.
KIT is grappling with the challenges of modern research infrastructure, particularly in containerization and data management. They are exploring solutions like Docker Swarm and Portainer to manage their containerized applications, aiming to improve efficiency, reproducibility, and knowledge transfer. The institute is also involved in larger German and European research initiatives, such as the National Research Data Infrastructure (NFDI) and the European Open Science Cloud (EOSC), positioning them as influential in shaping research practices and infrastructure on both national and international levels.
Key objectives
"You have to SSH into these machines and manually start Docker or start your container, stop your container, and there are very few people who are able to do that because most people here are scientists."
Addressing the Challenge
Ease of Use and Accessibility
Portainer significantly simplifies container management for scientists and researchers who may not have extensive technical expertise.
“You have to SSH into these machines and manually start Docker or start your container, stop your container, and there are very few people who are able to do that because most people here are scientists.”
Portainer addresses this issue because:
“With Portainer we can manage containers independently and it helps us to deploy the containers in a more convenient and more transparent way.”
The user-friendly interface allows researchers to manage containers without complex command-line interactions:
“If you have a container registry and we have one, you just connect Portainer to this container registry and you can pull the latest image of your container into Portainer and just run the container. You don’t have to do a single command on your command line and it doesn’t matter where all that stuff is actually located.”
Portainer enables more efficient workflows, particularly in near real-time processing:
“We are moving away from a retrospective approach where we are just analyzing data in the past to a much more real time focused approach.”
This improvement in workflow could lead to significant productivity gains. Eventually with 40-50 thousand researchers and 70-80% adoption of the containerization, managing it from a single source, would create a huge benefit.
Collaboration and Knowledge Transfer
Portainer facilitates collaboration across research centers:
“We see the infrastructure getting more and more harmonized. So that made it very easy for us to actually decide to also use Portainer because it helps to create synergies between the centers.”
It also supports “Praxis transfer”- the practical application and adoption of their research tools by end-users in real-world scenarios
“In the past, we had to go to Africa and install research tools on a server there and in most of the cases that never really worked because then something crashed and the people were gone. With Portainer, we just take a container and put it somewhere else.”
A couple of years ago, there was enormous flooding in Sudan that killed hundreds of people. At a conference later on, a water manager from Sudan said that he started using the forecast from KIT (Karlsruhe Institute of Technology).
“Since then we never had any problem with flooding again because we knew exactly what was coming.”
This demonstrates the real-world impact of their research and tools. It illustrates how the transfer of their forecasting technology to stakeholders in Sudan helped prevent loss of life and manage flooding risks more effectively.
Standardization and Reproducibility
Portainer promotes standardization and improves reproducibility of research:
“When we write a paper, we have to guarantee the results that we publish, that they are somehow reproducible and also in the same context. So if we compile a nice container that contains everything that is necessary that has a lot of potential for the reproducibility.”
Infrastructure Management
Portainer addresses the challenge of limited infrastructure resources:
“In Germany we are struggling with people in infrastructure. So in a Research Center, we usually do not have enough people in infrastructure.”
Portainer provides a centralized platform for managing containers across multiple environments:
“We would like to provide the people a platform where they can manage their containers independently and also a platform that helps us to deploy the containers in a more convenient and more transparent way.”
Security and Access Control
Portainer integrates with existing authorization frameworks:
“We can use the single authorization framework that is adopted across German research networks (OAuth) and connect our Portainer instances to this. I can easily integrate more people from our research environment to use our Portainer instance.”
It also provides role-based access control:
“I can give permission to work in a specific Portainer environment, or if a person is allowed to start a container or stop it.”
Cost-Effectiveness
Portainer offers and even encourages more manageable alternatives to complex Kubernetes setups:
“We somehow decided for our institute that we don’t want to go that route because we just don’t have the resources to to maintain a full release instance. So we were rather aiming for a for a smaller Docker Swarm cluster.”
Flexibility and Portability
Portainer supports different container environments and facilitates the transition to cloud-native approaches:
“You have your infrastructure, you don’t care where this is located. You can have have it somewhere in your institute and you have your storage somewhere. Storage could be here in the institute. It could be somewhere completely else. And you just communicate via URLs and that makes it very transparent and very reproducible.”
This flexibility allows for easy deployment across various platforms:
“I can just take a container, run it here on our Portainer or put it at the other Research Center. The data will be at the same location. You don’t have to care where exactly.”
"We can use the single authorization framework that is adopted across German research networks (OAuth) and connect our Portainer instances to this. I can easily integrate more people from our research environment to use our Portainer instance."
Metrics
Portainer has had a significant impact on building out a modern framework that improved operations for researchers within KIT and sets an example for environmental researchers throughout Europe. Key metrics include:
- 10,000 researchers under the Helmholtz Association
- Thousands of lives saved or impacted by the adoption of KIT containers for weather forecasting, by Ethiopian Weather Service
- 10-15% impact on funding of projects due to reproducibility, and product oriented outcomes made possible with containerization.