Empowering Research with Advanced Gas Monitoring
Eosense, a company specializing in environmental monitoring, approached Éphémère Creative to create a user-friendly dashboard for their clients. The dashboard needed to provide real-time data visualization, role-based access, and the ability to create and manage “jobs” tied to specific devices and timeframes. Éphémère Creative designed and developed a scalable and efficient solution to meet Eosense’s requirements.
The main challenges faced by Éphémère Creative were:
Regularly fetching and processing data from FTP servers without duplicating old data
Efficiently Assigning and Streamlining User Permissions Management
Visualizing and annotating multiple metrics for data analysis
Ensuring low hosting costs while allowing for future scalability
Data Retrieval and Processing
Éphémère Creative built a backend system to pull data from the FTP servers at regular intervals. Initially, data was fetched every five minutes, but the rate was later reduced to every 30 minutes or an hour as per the FTP server’s request. The system was deployed on a Kubernetes cluster using CronJobs to schedule the tasks. A Python script was developed to import the data and prevent duplication.
Role-based access and job management
The dashboard allowed the account owner or administrator to create “jobs” for specific devices and timeframes. Team members could be assigned to work with the data from different jobs based on their access permissions. Metadata related to each job, such as client information and location, could also be added.
data visualization and annotation
Users with access to a specific job could plot multiple metrics from the devices, such as methane emissions, and view data in various aggregations (e.g., average, maximum, or minimum). The dashboard allowed users to add annotations to specific metrics at specific times, providing a powerful tool for in-depth data analysis.
The following technologies were used to create this solution:
Kubernetes and CronJobs
for data retrieval and processing
for API and permission management
on a managed Postgres instance with Digital Ocean for hosting
for the frontend, using Material-UI (MUI) for design
for API communication
for migrations and providing an admin interface for Eosense
The custom dashboard created by Éphémère Creative was well-received by Eosense and its clients. The end users found it easy to access and analyze data from various devices, jobs, and metrics. The system proved to be efficient, cost-effective, and scalable, allowing Eosense to manage and visualize their environmental monitoring data with ease.