Farmers Dashboard Toolkit

(1)

Farmers Dashboard Toolkit

Adaptive and configurable dashboard for Big Data

Javaria Hazoor

Thesis submitted for the degree of

Master in Programming and System Architecture 30 credits

Department of Informatics

Faculty of mathematics and natural sciences

UNIVERSITY OF OSLO

(2)

(3)

Farmers Dashboard Toolkit

Adaptive and configurable dashboard for Big Data

Javaria Hazoor

(4)

Printed: Reprosentralen, University of Oslo

(5)

Abstract

With the digitalization of the agri-food sector, the flow of the data in this domain becomes more and more important, especially for the farmers. Today’s data driven digital solutions for agriculture pose challenges for the farmers: the existing solutions do not inter- operate well with each other and do not always capture good enough individual farmer’s needs. The farmer has to cope with a wide variety of apps, ICT programs, farm management systems, digital solutions and dashboards and all core actors in the agri-food domain will want to collaborate with farmers by having farmer’s attention and “clicks/likes/touch”. The farmer typically gets overwhelmed in a complex digital landscape and missed essential information and the overall picture for his farm management.

The purpose of this thesis is to design and develop a solution for a self-service dashboard for farmers so that farmers can configure their own dashboard according to their own needs and requirements without any help from developers. This dashboard of dashboards called as

“Farmers Dashboard” is aimed to provide a better overview of the farm activities, productivity, timely warnings, alerts and decision support system to the farmers.

The proposed solution also aims to give support to app providers by not restricting them to the specific libraries, framework and technologies but giving them liberty to re-use their existing user interface and export it as a web component to the dashboard.

(6)

Acknowledgements

I would like to express my gratitude to everyone who contributed to the process of writing my thesis.

First of all I would like to thank the advisor team atSI NTEF with Arne Jørgen Berre, Bjørn Marius von Zernichow for their valuable guidance, patience, engagement, motivation, and contributions to the scientific and technical discussions, ideas, remarks, thesis research and writing. I am also very thankful to Dumitru Roman (supervisor at UiO) for his valuable feedback on the thesis.

Secondly, I would like to thank the Norwegian DEMETER project partners Landbruket Dataflyt and Mimiro for their collaboration and discussions in the implementation and evaluation part.

Last but not the least I would like to thank my parents and my sib- lings for the support and encouragement throughout this period.

Javaria Hazoor December, 2021

(7)

List of Figures

2.1 Big data vendors and technologies . . . 10

2.2 Technology stack for Big Data . . . 10

2.3 Big Data workflow . . . 11

2.4 Big Data sizes . . . 13

2.5 Overview of the main DEMETER concepts . . . 16

2.6 High-level view of DEMETER Reference Architecture instantiation example . . . 19

2.7 DEMETER Reference Architecture mapped to Big Data and AI Pipeline . . . 21

4.1 Bit.dev micro-frontend framework [31] . . . 36

4.2 Process of development with Piral [7] . . . 38

5.1 Overview of micro-frontend architecture [23] . . . 45

5.2 Wireframe of the first version of the Farmers Dashboard MVP . . . 46

5.3 Architecture of the first version of the Farmers Dash- board MVP . . . 47

6.1 Global context information . . . 50

6.2 Farmers Dashboard configuration of dashboard layout . 51 6.3 Adaptation management system [25] . . . 59

6.4 Farmers Dashboard MVP with integrated web-componets in different frontend technologies . . . 62

(11)

List of Tables

4.1 Assessment of existing frameworks . . . 34

4.2 Evaluation of micro-frontends . . . 40

6.1 Assessment of AU I approaches . . . 61

7.1 Evaluation of Farmers Dashboard Toolkit . . . 68

(12)

List of Abbreviations

AIM Agriculture Information Model AIS Agricultural Interoperability Space

AKIS Agriculture knowledge information systems AUI Adaptive User Interface

BI Business intelligent

BSE Brokerage Service Environment

DEH DEMETER Enabler HUB

DEMETER Digital Electronic Mapping of European Territory DSS Decision support system

EIP Entrepreneurship and Innovation Programme

IDSA International Data Space Association IoT Internet of Things

(13)

LD Landbrukets Dataflyt

MVP Minimum viable product

SINTEF Selskapet for INdustriell og TEknisk Forskning ved norges tekniske hoegskole (The Foundation for Sci- entific and Industrial Research at the Norwegian In- stitute of Technology) [1]

SOCS Stakeholders Open Collaboration Space

UI User interface

(14)

Part I

Introduction and Background

(15)

Chapter 1 Introduction

This thesis has been developed in the context of the European DEMETER project [2] aiming at Digital Platform Interoperability within the Agri-Food sector, with a particular focus on design and realization of a Farmers Dashboard. As a short master thesis during the fall of 2021 the focus has been within a limited part of this project, focusing on interoperability and integration of user interfaces.

The context for this thesis is to represent the bigger picture of the data by creating an adaptive and configurable dashboard for big data considering the farmers dashboard as a case study in the agricultural sector.

The data flow for the farmers is a big challenge and opportunity for business development in the sector. Today’s digital solutions for agriculture do not communicate good enough together and are not largely based on the individual farmer’s needs. The farmer has a lot of apps, ICT programs, farm management systems, digital solutions and dashboards to cope with and all actors will want to collaborate with farmers by having farmer’s attention. The farmer gets overwhelmed in a complex digital landscape and missed essential information and the overall picture for his farm management.

1.1 Motivation

The introduction has presented the need for a Farmers Dashboard and as the data is growing exponentially with each passing day,

(16)

the representation and visualization of big data are also getting complicated which is increasing complexity and dense information to the dashboard user interfaces and its usability tends to decrease. The main challenge of user interface design is to create solutions that are usable, useful, and trustful by users.

Hence, there is a need for software development approaches and frameworks that allow information to be presented in a simple and concise manner. Thus, the proposed approach will evaluate different technologies for developing the adaptive and configurable Farmers Dashboard Toolkit that will integrate various stand-alone dashboards, farm management systems, app’s and ICT-programs for the farmer in a common framework and the aim of the solution is to:

• Provide essential decision support for individual farmer and more specific farm management systems which the farmer use seeking to optimize their farm’s productivity.

• Streamline the digital everyday life for the farmer and try to get better collaboration between different websites, applications, programs etc. that are used by the farmers.

1.2 Research questions

The scope of the thesis is to develop a dashboard toolkit that will create a configurable and adaptive dashboard for farmers which will provide them a configurable decision support system. Based on this following are the research questions that needed to be answered by the end of this thesis

1. How to develop an adaptive and configurable Farmers Dashboard Toolkit for farmers using Big Data and IoT data and which user interface technologies can help us to achieve the thesis goal?

2. Can the Farmers Dashboard Toolkit be used in order to create an adaptive,configurable and integrated dashboard by being configured with existing components like Planting rate analysis from John Deere, Climate calculator from Landbruket Dataflyt, Milk prediction from Mimiro and many more?

(17)

1.3 Main objective

Based on the research questions presented in the previous section following are the main objectives of the research work.

Objective 1: To study and compare possible different frontend technologies and frameworks to select the suitable technology.

Objective 2: Develop an approach that can summarize large amount of input data into manageable chunks of visual information and allows the user to make informed decisions that impact business performance and creates added value.

Objective 3: Using the visualization techniques and frameworks develop the software prototype of an adaptive and configurable dashboard with maximum usability.

Objective 4: Evaluate the approach by using a relevant use case pilot in a current DEMETER innovation project.

1.4 Thesis contribution

This thesis work provides the foundation of a Farmers Dashboard Toolkit which will be used to create a dashboard of dashboards called Farmers Dashboard which will be considered as an innovation toolkit where third-party app provider will publish their components or applications that will be used by farmers to create their own customized digital dashboard for a better view/outlook over the farm activities and provide the farmer’s cooperation with both private and public actors which will be a more efficient use of digital tools that has been applied by the farmer and providing the better and more customized decision support for each farmer.

1.5 Research methodology

The research method for the development and evaluation of an adaptive and configurable dashboard is based on the approach and

(18)

scientific methodology of Technology research, presented in a report by Ida Solheim and Ketil Stølen which defined Technology research as

Technology research is the research for the purpose of producing new and better artifacts[37]

This report elaborates the technological research and ways to conduct it. Making a new artifact or improving existing artifacts require requirement analysis after which hypothesis is defined and new artifact is evaluated based on predictions. Technology research is an iterative process where the following three steps are repeated again and again if the evaluation concludes that the results are not fulfilling the requirements. All of these steps are taken during the development of Farmers Dashboard Toolkit.

Problem analysis The potential need for a new artifact is identified and try to understand the challenges, examine and evaluate the existing solutions to the problem.

Innovation A new artifact that satisfies the need that is constructed and finding a specific, scoped down the area where a contribution can be made.

Evaluation Evaluate the artifacts and innovation that is formulated against the requirement that has been identified in the phase of problem analysis

1.6 Work behind the project

The thesis work was carried out as part of the DEMETER project at SI NTEFwhich will serve the basis of the concept of a configurable and adaptable dashboard and its integration with this project. This will be considered as an innovation toolkit where third-party developers can publish applications and content for a specific part of the farm and the land.

This digital dashboard will focus basically on a better view and overlook over the farm activities and farmer’s cooperation with both private and public actors, will provide more efficient usage

(19)

of digital tools and also providing a better and more customized decision support system for each farmer. In the pilot, SI NTEF and Landbrukets Dataflyt leads the work with developing the broad scope dashboard and will build data infrastructure and models of farmers dashboard in a co-creational process based on the agricultural data flow infrastructure today, especially related to the companies authentication and authorization systems, with farmers, related partners and industries in Norway and Europe.

The forecasting models for dairy farms are essential to plan and optimize the production in terms of economy, animal numbers, milk quality and feed production. For this reason, forecasting models must be developed at different aggregated levels ranging from single animals, herd groups, herd level, up to regional and national levels.

This way farmers will be provided with an integrated 360-solution application to optimize farm production.

1.7 Thesis structure

This section briefly describes the structure of the report and provides an overview of the content discussed in each of the chapters.

Chapter 1 – Introduction outlines the general idea and states the motivation behind the concept of Farmers Dashboard and why we need to create a configurable and adaptive dashboard for farmers.

Chapter 2 – Background describes the background knowledge about the visualization of big data andIoTand the challenges are emerging in the visualization of big data with the passage of time with the consideration of data flow in the agri-food sector.

Chapter 3 – Requirements for the Farmers Dashboard Toolkit for the solution highlights the need for the farmers dashboard and defines the set of requirements to fulfil the needs of the end-user as well as app developers.

Chapter 4 – Assessment and selection of existing frameworks and technologies explores the existing frameworks, tools and

(20)

technologies and comparing with each other to evaluate that which framework or technology will be the best to develop the Farmers Dashboard Toolkit.

Chapter 5 – Design and architecture of Farmers Dashboard Toolkit Practical aspects of the modern concept of an adaptive and configurable dashboard and its usability along with theoretical knowledge about developing these types of dashboard are presented in this chapter, which are vital for understanding the rest of the thesis.

Chapter 6 – Implementation of Farmers Dashboard Toolkit provides the basis of implementation of the dashboard which is based on dashboard architecture and its design approach in which the dashboard will be decomposed into individual, semi-independent micro-apps which will be loosely working together.

Chapter 7 – Evaluation validates the usability of the configurable and adaptive dashboard and to what extent the defined requirements has been achieved and fulfilled.

Chapter 8 – Conclusion and future work summarizes the evaluation in accordance with the requirements and proposes future research opportunities within the research field of visualization of big data.

(21)

Chapter 2 Background

This chapter provides an overview and background knowledge about the concepts, technologies and theories that are vital with reference to the thesis work. The chapter provides an introduction to Big Data and the big data technology stack and focusing mainly on its visualization and open challenges.

2.1 What is Big Data

Gartner, Inc., an American IT research and advisory firm, defined Big Data as high-volume, high-velocity and/or high-variety information assets that demand cost-effective, innovative forms of information processing that enables enhanced insight, decision making, and process automation [16].

The emerging technological development of big data is recognized as one of the most critical areas of future information technology and it is evolving at a rapid speed, driven in part by social media and the IoT phenomenon. The technological developments in big data infrastructure, analytics and services allow firms to transform into data-driven organizations. Due to the potential of big data becoming a game-changer, every firm needs to build capabilities to leverage big data in order to stay competitive [29].

Essentially, big data has the following salient features called “4Vs”

differentiating it from other concepts, such as “very large data,” “large- volume data,” and “massive data.”

(22)

Volume: The quantity of generated and stored data usually refer to the data volume from terabytes to petabytes

Variety: The type and nature of the data (structured, semi-structured, unstructured, text and multimedia)

Velocity: The speed at which the data is generated and processed to meet the demands (e.g., real-time).

Value: The analytical results based on big data can bring huge both business value and social value [20]

2.1.1 Technology stack of Big Data

The giant network of connected things is growing continuously, creating an influx of data. Different tools and technologies are available in the IT industry, to capture and analyze huge masses of data. Organizations are taking the data usage to new levels as they are acknowledging the value created by using the data-at-rest (archived) and data-in-motion (streaming) simultaneously for even better performance and availability. Big data technology is defined as

Analyze, process and interpret the massive amount of structured and unstructured data that could not be processed manually or traditionally[41]

There are three main characteristics of Big Data: the data itself, the analytics of the data and the presentation of the analytics results.

Then there are the products and services that can be wrapped around these Big Data elements to address the various challenges related to Big Data, including capture, storage, querying, updating, processing, analysis, searching, sharing, transfer, visualization, and information privacy. These products and services are collectively referred to as the Big Data technology stack.

Many technologies are continuously emerging in both the propri- etary and open-source industry. Figure 2.1 shows big data vendors and technologies. The decision to choose the appropriate set of these products for a given use case, depends on several factors, majorly:

• Data specific: type, magnitude and speed of data to be handled

(23)

• Application type: capture, storage, querying, updating, processing, analysis, searching, sharing, integration, transfer, visualization, information privacy.

Figure 2.1: Big data vendors and technologies

The generalized technology stack for Big Data architecture, where different tools can be categorized and mapped with reference to big data areas where they are applicable to is shown in the figure 2.2.

Figure 2.2: Technology stack for Big Data

(24)

2.1.2 Big Data workflow

A Big data workflow is a unified system of a group of logically related activities at performing a particular task for capturing events for analysis and building products. Each activity in a workflow defines a specific action to perform on the data. Each activity can take zero or more data sets as an input and can produce one or more data sets as output.

Big data workflow consists of the following steps, as illustrated in figure 2.3 and these steps are defined as:

Data Collection — Structured, unstructured and semi-structured data from multiple sources and loading vast amounts of data onto a single datastore

Data storage and preparation — Discovery, Cleansing and understanding format and content and linking, entity extraction, entity resolution, indexing and data fusion

Analysis — Intelligence, statistics, predictive and text analytics, machine learning

Delivery — Querying, visualization, real-time delivery on enterprise- class availability [18]

Figure 2.3: Big Data workflow

2.2 Big Data visualization

Data visualization is the presentation of data in a pictorial or graphical format, and a data visualization tool is the software that generates this presentation. It provides users with intuitive means to interactively explore and analyze data, enabling them to effectively

(25)

identify interesting patterns, infer correlations and causalities, and supports sense-making activities [17].

Data visualization and analytics are nowadays one of the corner- stones of Data Science, turning the abundance of Big Data being produced through modern systems into actionable knowledge. Indeed, the Big Data era has realized the availability of voluminous data sets that are dynamic, noisy and heterogeneous in nature. Transforming a data-curious user into someone who can access and analyze that data is even more burdensome now for a great number of users with little or no support and expertise on the data processing part. Thus, the area of data visualization and analysis has gained great attention recently, calling for joint action from different research areas from the Information Visualization, Human-Computer Interaction, Machine Learning, Data management and Mining, and Computer Graphics.

Dealing with large volumes of data, often leads to chaos and misinterpretations between the data statistics, thus visually grouping together with many data points significantly enhances the quality of the data analytics and provides a much convenient way for the business executives, data scientists in understanding the relationships between the data. Interpreting the data visually helps in understanding the data and quickly deciding where to focus the research.

2.2.1 Conceptualizing the Big Data visualizing for understanding the future need

Without having all the data in front, it is difficult to see the big picture.

One way to solve such a problem is through data visualization using an intelligent dashboard where all the necessary information to make a quick decision can be visualized right in front of their eyes. From the dashboard, it is hoped that a wider range of data from different sensors can communicate and predict the risk and urgent issues. At the same time, it helps to increase the awareness of the latest situation and to plan for what to do next. It can also be a platform to reflect on other major needs in the past and learn some invaluable lessons from them.

And lastly, to serve as a rich database for decision-makers to make important decisions when another emergency calls.

(26)

In this thesis, the data from agriculture technologies and the use case of farmers dashboard has been considered to provide the solution which will be based on re-usability of existing user interfaces, scalable and providing the decision support system to each farmer according to their own needs and farm.

2.2.2 Open challenges in Big Data visualization

For decades, businesses have collected data, analyzed it using a variety of Business Intelligence (BI) tools, and generated reports. The process continued for a long duration of time, but eventually, a few highly trained data analysts were able to pull the desired results and figures.

Businesses are finding that this traditional reporting process does not work nearly as well for big data and certainly is not sufficient to capture the potential value of the big data. Therefore, it represents both a challenge and an opportunity. The challenge is related to how this volume of data is controlled, and the opportunity is related to how the effectiveness of this data is enhanced by properly analyzing the information.

Figure 2.4: Big Data sizes

Data analytics and visualization are not new. The primary challenges stem from what is commonly termed the “three Vs” of big data: volume, variety, and velocity. Most traditional reporting and data mining tools cannot handle the vast volume of big data-although the variety and velocity of the data often present even greater challenges.

(27)

Thus visually representing the data has made the big data much more adaptable and valuable [22].

2.2.3 Challenges for the visualization of farmers and agricultural data flow

According to the United Nations, the world population will reach approx. 10 billion by 2050. Global agriculture production is expected to rise 69% between 2010 and 2050 [36]. Increased pressure on food demand, together with climate change, is driving the agri-food sector to massively adoptIoTtechnologies which bring the promise of not only improving productivity but also increasing sustainability and safety.

For many years, deploying IoT in the agri-food sector has proven to be difficult despite market demand and a large number of use cases that would benefit from IoT. Connectivity coverage, cost, availability of purpose-built sensors and actuators etc., are among the market inhibitors which prevented massive market uptake for some time.

However, many of these market barriers are being overcome with outcome-based solutions being massively deployed for areas such as the use of resources, monitoring of animals, the use of robotics for farming, etc. [26]. Their value propositions are about increased productivity, increased safety, food trace-ability and saving the environment.

2.2.4 Role of DEMETER in the European agri-food sector

The Main Concepts of DEMETER

DEMETER is an H2020 project launched under DT-ICT-08-2019 focusing on Building an Interoperable, Data-Driven, Innovative and Sustainable European Agri-Food Sector. It is built around the four main concepts presented below:

1. The DEMETER Stakeholders Open Collaboration Space (SOCS), focusing on resolving the demands on the farmer’s side using a structured process that converts an individual need or the most relevant/shared need from a set of needs to a

(28)

challenge. A challenge is then resolved through a unique co- creation process, in which farmers, service advisors and providers can select, together, the most appropriate set of tools, devices, components, data sources, etc., taking into account the existing ones already deployed at the farmers and the farmer-defined improvement goals. The DEMETER SOCS also includes a wide range of features that, together, deliver the knowledge sharing and improvement process, structuring the human-in-the-loop dimension of DEMETER. Furthermore, the DEMETER SOCS is strongly inspired by the EIP Agri Social Spaces and Operational Groups, operating as a set of defined activities for multiple actors implemented through physical meetings, workshops, hackathons, etc., and supported by a dedicated online facility.

2. The DEMETER Agricultural Interoperability Space (AIS), which focuses on delivering a full set of mechanisms and tools intended to support the interoperation, integration and deployment of solutions. DEMETER does not define completely new interoperability mechanisms but uses (and extends) a wide range of pre-existing tools at device, data, service and application levels. Data in AIS are mainly shared either through the integration with data spaces and other data services or through pub/sub channels. Standardized interfaces and data models are offered to support expandable agriculture applications. AIS supports deployment to cloud, datacenter, hybrid, edge and other environments.

3. The DEMETER Enabler HUB (DEH), which centralises the full description of all the components, devices, services, data sources, platforms etc that are accessible for deployment. The HUB provides, on the one side, the harmonised description that enables each component to be used in the co-creation mechanism, and on the other side, its uptake in different deployment through the full set of DEMETER enabled interoperability mechanisms.

The DEMETER Enabler HUB includes the mechanisms to ensure interoperability with components provided through other initi- atives, such as IOF2020, DATABIO, AFArCloud, SmartAgriHub

(29)

Figure 2.5: Overview of the main DEMETER concepts etc.

A large set of available components will be available for integration and configuration into the Farmers Dashboard – through the DEMETER Enabler Hub.

4. Brokerage Service Environment (BSE) is a core module of DEMETER architecture, which facilitates the registration, discovery and ultimately communication process for the DEMETER- enabled resources in a secure and privacy preserving manner.

5. Access Control System (ACS) is the component in charge of managing access control to DEMETER Cloud resources in a secure and efficient way.

6. The DEMETER Dashboard, the sole entry point to the DEMETERecosystem for allDEMETERStakeholders. The Dash- board also offers user friendly interfaces to access, understand and control data related to their own accounts, to perform basic administration tasks over their DEMETERaccounts, get an overview of the usage of their business data by external stakeholders, etc. The farmer´s dashboard focused on here will become an addi- tion to this – focusing directly on the decision support for a farmer.

A high-level overview of these concepts is presented in the figure 2.5 the key benefits is connecting a human-focused interaction space with

(30)

the actual digital implementation space. This ensures that DEMETER remains fully human-centric and human-driven – delivering digital enablers that are fully aligned to the farmers’ needs and based on the knowledge and wisdom captured through structured mechanisms.

The DEMETER dashboard will include this Farmers Dashboard in future. The DEMETER Enabler Hub acts as a point of reference for the interested developers and stakeholders in order to register their offered capabilities and resources and act as DEMETER Providers.

These offerings are semantically described and are escorted by metadata, which include the security and data usage policies applicable, Quality of Service metrics, etc. DEMETER Consumers can browse the DEMETER Enabler Hub to discover suitable capabilities and resources matching their requirements and specified criteria. The Hub verifies the identities of the consumers and the providers and provides the support necessary for the establishment of a direct secure communication channel among them. The Enablers, i.e., the basic building blocks to build DEMETER-enhanced systems, will be made available via DEMETER Enabler Hub, being either services developed by the project or resources offered by external stakeholders, i.e., by third party (3P) service providers, or by platform providers.

Finally, it is highlighted that any platform, thing, service or application that makes itself available via the DEMETER Enabler Hub, or consumes resources available in the Hub, or both, is represented by a DEMETER-enhanced Entity. In order to allow for full scale secure interoperability and communications, there are a few specific DEMETER enablers that are mandatory and need to be available at each DEMETER-enhanced Entity. These are the DEMETER Core En- ablers that are encapsulated, along with the DEMETER Provider and Consumer in theDEMETEREnhancing Service. The Entities are com- municating with the DEMETERHub via the DEMETERHub API. All DEMETER-enabled IoT platforms and individual Thing/resources are registered in theDEMETERRegistry, along with access rights/policies.

The various instantiations of the DEMETER-enhanced entity are a core part of the DEMETER Reference Architecture elaborated upon in the next section.

(31)

DEMETER Reference Architecture

This section aims to provide a high-level view of the DEMETER Ref- erence Architecture. There are a variety of smart farming systems and platforms already deployed, employing many different communication, sensing, and data processing technologies. However, building a new master system that has the ability to also incorporate other existing systems is a near impossible task, also due to the complexity/heterogeneity of the agri-food sector when it comes to issues such as scalability and governance. To tackle these, DEMETER offers an overarching approach that integrates heterogeneous technologies, platforms and systems, while supporting fluid data exchange across the entire agri-food chain, addressing scalability and governance of ownership. These goals are delivered through the Agricultural Interoper- ability Space (AIS). The proposed approach enables existing agriculture knowledge information systems (AKISs) to continue their operation, but also allows those systems to make available and consume data from other cooperating systems. Additionally, newer technologies and services can be exposed that may be of interest to cooperating AKISs.

This is more realistic and viable in terms of usability, market adoption, and sustainability. In order to realize this approach and address the extracted requirements, the following two core objectives need to be fulfilled by the proposed solution: (i) allowing existing AKISs to offer their data to and consume data from their counterparts, providing also the means to incentivize AKISs for sharing data by ensuring data integrity and valorisation, giving them also the chance to make some profit, (ii) ensuring rapid deployment, portability and scaling once required via using virtualization containers for the various services.

The proposed architecture is presented in figure 2.6 and is built on DEMETER Provider and DEMETER Consumer services, based on the architectural model introduced by the International Data Space Association (IDSA). However, DEMETER Provider and DEMETER Consumer services further extend their applications by supporting AKISs to also expose and consume data. Rapid deployment and decommissioning are highly beneficial for survey services that might not require a continuous feed from a particular AKIS. Such a service would deploy and start a DEMETER Consumer service for

(32)

Figure 2.6: High-level view of DEMETER Reference Architecture instantiation example

(33)

that particular AKIS, gather necessary information, and then stop the service. The service is packaged into a lightweight container along with all the software necessary to support self-contained deployment of the service (runtime environment, libraries for supported communication protocols, encryption techniques, etc.).

Since data interoperability is of critical importance, the proposed solution provides the necessary data translation mechanisms combin- ing the use of a semantic data model (Agriculture Information Model

— AIM) developed by DEMETER, along with the respective data translation/management/inference mechanisms adopting widespread standardised solutions such as NGSI-LD, Saref4Agri, ADAPT, etc. In order to enable interoperability of heterogeneous data handling approaches, theDEMETERprovider-consumer services, deployed on various AKISs, translate and exchange data based on the AI M common data format with the utilization of lightweight data wrappers/translat- ors. For this conversion to be feasible, each AKIS needs to provide the specifications of the utilized data model-semantics, or it should parse returning content in AI M format. The AI M incorporates and extends existing ontologies and vocabularies already available for this domain. DEMETER provider-consumer services maintain the necessary mechanisms for addressing data security and privacy concerns.

They need to be trusted to be deployed and hosted by the AKISon their own cyber-premises (i.e., hosting environments) following the principle that moving processing capabilities is easier than moving data them- selves. This is also an inherent data privacy protection feature as the owner of the data maintains the control/decision of which data is allowed to be shared with other entities. The services need to provide privacy and security functionalities, including user authentication and access authorization. Once a DEMETER-enabled application is implemented, the final version at a production level can be discovered by consumers (e.g., Farmers, Agronomists, Cooperatives, etc.) through the DEMETER Dashboard, which is also used by these stakeholders to provide their feedback regarding the perceived experience and added value.

For simplicity, figure 2.6 presents only some of the platforms that are integrated into the DEMETER Reference Architecture, thus

(34)

Figure 2.7: DEMETER Reference Architecture mapped to Big Data and AI Pipeline

representing a specific instantiation of the architecture deployed to serve the needs of one pilot site for example. However, apart from platforms, DEMETER service logic blocks are made available and can be used by the interested parties (e.g., data/knowledge facilities), as well as any other 3rd party resource. All the registered resources are made available to the developers through the DEMETER Enabler Hub; these are annotated with rich metadata that describes the capabilities (or constraints) of these resources, thus guiding the deployment of DEMETER apps based on the adopted technologies as well as information regarding ownership of resources that are available and the restrictions that their locations might impose during this process. Each platform, thing, service or application is represented by a DEMETER-enhanced Entity and either makes itself available via the DEMETER Enabler Hub or consumes resources available in the Hub, or both. Finally, there are various instantiations of the DEMETER-enhanced entity (i.e., the DEMETER-enhanced Resource, the DEMETER-enabled Service and the DEMETER-enabled Application) [3].

Figure 2.7 shows the DEMETER Reference Architecture related to the Big Data and AI Pipeline framework described in section 2.2.

(35)

Part II

Requirements and Assessment of existing

framework and technologies

(36)

Chapter 3 Requirements for the

Farmers Dashboard Toolkit

In this chapter, we had analyze and discuss several aspects that will eventually lead to the set of requirements for the implementation of the prototype that will actually represent the Farmers Dashboard.

The data flow for farmers is a big challenge and opportunity for business development in the sector as today’s digital solutions for agriculture does not speak good enough together and are not largely based on the needs of the individual farmer. The farmer has a lot of applications, farm management systems, digital solutions and dashboards to cope with, and all actors the farmer collaborate with want to have a part of the farmer’s attention by “clicks,likes and touch”. In this complicated digital landscape, the farmer often gets overwhelmed and missed some important information, warnings or alerts. So the requirements emerge from this need of farmers and it is the research opportunity to propose an approach that will contribute to improve the farmer’s agriculture system but also provide a dashboard toolkit to app providers to keep on adding value to the Farmers Dashboard by putting minimum efforts.

DEMETER aims to empower farmers and farmer cooperatives in two main ways. First, by allowing farmers to use their existing machinery and platforms to deliver new, integrated knowledge to help decision-making. Second, by easing farmers’ updating or acquisition of machinery, platforms and sensors ensuring technologies speak a common language meaning they can be easily connected, combined and

(37)

cooperate with each other.

3.1 Vision and approach for the Farmers Dashboard

The vision behind Farmers Dashboard is to create a digital dashboard which will be scalable, configurable and adaptive according to each farmer’s requirements and it will provide a better view over the farm activities and the farmer’s cooperation with both private and public actors and provide more efficient use of digital tools the farmer applied, and also providing the better and more customized decision support for each farmer by adding new and exiting components into the Farmers Dashboard Toolkit by the DEMETER Enabler Hub (DEH). Due to shortage of time our primary focus is on limited areas of farms in the pilot, but the concept will include all types of farms. The vision for Farmers Dashboard Toolkit as support in realizing dashboard is that it should

• Add value to the farmers

• Self service configurable dashboard

• Easy to maintain

• Easy to expand

• Reusing existing user interfaces

3.2 Requirements for the Farmers Dashboard Toolkit

The farmers dashboard has two types of users one will be the end-user i.e. farmer and the other type will be third-party app provider that will publish their apps as web components to Farmers Dashboard Toolkit.

So with respect to these two users we had split the requirements into two types which will be used to evaluate the existing dashboard

(38)

frameworks in next chapter and will be further used in chapter 7 to evaluate our dashboard toolkit. So, following are the requirements that will provide the foundation during the process of design and implementation of Farmers Dashboard Toolkit to create a dashboard of dashboards for each farmer.

3.2.1 End-user requirements

End-users will be the farmers that will use this toolkit to configure their own dashboard according to their own requirements and used machineries, sensors, platforms and technologies . So following are the requirements that need to be fulfilled by the toolkit so that it should meet the requirements of farmers.

Requirement 1: Increase use of own data flow Farmers often had multiple support systems from various providers that together has a very extensive data material and today’s solutions do not speak enough together to provide the bigger picture of the farm at one place. So, Farmers Dashboard Toolkit should provide a solution to the farmers that will provide increase use of their own generated data flow and get information of all the activities hap- pening in their farmhouse.

Requirement 2: Dashboard decision support In todays vast digital landscape of agri-food sector. Farmers often missed the overview of whole farm and the impact of the situation on the farm production. This happens due to the missing of the important information. The majority of the software products agriculture and farmers utilize various sensors to collect data which results in a large amount of data [35]. So, by taking advantage of this data the farmers dashboard is intended to provide the digital decision support system which will provide a better understanding according to each farmer’s needs for decision support and provide the streamline of the digital everyday life and strategic decision support system to the farmers with better collaboration between different apps, programs and websites.

Requirement 3: Overview and comparison with industry Farmers

(39)

need to have a better overview of his farm production and comparison to the other farms in the industry.

Requirement 4: Prediction for planning purposes The farmers dashboard should contain good forecasting models for production to be able to plan and optimize production in terms of economy, animal numbers, milk quality and feed production and get plenty of benefits for production including revenue generation, solid degrad- ation control, air quality and growth [27]. Farm-related organ- isations will get benefits of prediction for planning, economic and farm growth purposes during and after the pilot period.

Requirement 5: Net economic benefits Farmers that try the new decision support system by the Farmers Dashboard Toolkit will get a better overview and forecast of their land and farm that help will to get net economic benefits.

Requirement 6: Self-service configurable dashboard This is one of the most important functional requirement of the Farmers Dashboard Toolkit that the dashboard created for each farmer should be configurable so that is it easy for each farmer to set up the dashboard with the various components that are relevant for each farmer and they can configure or upgrade their dashboard without any help from the developers.

3.2.2 Third-party app providers requirements

This section contains all the requirements related to the service providers environment and requirements from the application developers point of view. The main goal is to achieve the maximum reusability of existing user interface and all the applications published in Farmers Dashboard Toolkit should be independent to each other so that they can be configured independently by each farmer.

Requirement 7: Cross platform integration As the modern world has a lot of modern technologies and frameworks so it should allow the developers to choose any programming language, technology and construction methods to develop the application

(40)

module. This will allow the app provider to choose the tools they have the most expertise and experience with and they should not need to learn new technologies in order to publish their app in the Farmers Dashboard Toolkit. This is an important requirement in the context of the dashboard toolkit to provide the support of cross-platform development.

Requirement 8: Reuse of existing user interfaces Software re-usability is an attribute in which software or its module is reused with

very little or no modification. Software re-usability offers the great potential of significant gains for an organization by reducing cost and effort and accelerating the Time to Market of software products [44]. The agriculture and farmer’s world is full of huge amount of software products. So in order to avoid the recreation of the same functionalities again specifically for this platform, the compatibility to the existing solutions should be provided and the app providers should be able to import the components from their existing project code to the Farmers Dashboard Toolkit.

Requirement 9 : Decoupling of the components All the components integrated into the platform should be independent of each other. A decoupled system consists of one or more stand alone computing components or elements. Each component is connected via shared interface but can operate independently. Their performance and operation usually does not affect other systems [21].

Requirement 10: Extensible dashboard Platform extensibility means that it can be extended without modifying the original codebase.

So, the developers add to the base functionality, thereby offering new capabilities and outputs and allowing the app providers to plug in other more desired systems without re-configuring the entire factory [14]. The platform should be extensible so that app providers keep on adding value to the platform as well as its farmer users to keep on upgrading their own dashboard.

Requirement 11: Cost benefits are the core benefit for any business so this is a non-functional requirement of the platform that

(41)

the personalized dashboard configured for each farmer should be able to give cost benefits and this cost-benefit is with respect to the service or app providers.

Requirement 12: Development time This platform should not be only to provide the services to the farmers but also to provide ease to the developers. The development time should be reduced or very minimal so that it is very easy and attractive to the software product owners to publish their app on the Farmers Dashboard Toolkit to gain maximum benefits with the least effort.

Requirement 13: Simplicity The integration of application by the third party software companies should be simple and easy to integrate.

Requirement 14: Single login As the purpose of this toolkit is to configure a separate dashboard using the existing application interfaces and the collaboration with different software companies in this sector but the farmer should be able to login only one time and he should be able to access all features.

(42)

Chapter 4 Assessment and selection of existing frameworks and

technologies

Creating a dashboard is not a new concept but the data flow for farmers is a big challenge and opportunity for business development in this sector. So in this chapter, assessment of different technologies and frameworks will be done which can be used to create Farmers Dashboard . Firstly, tools, technologies and frameworks are presented and then the evaluation method is defined which will be used to evaluate against the requirement defined in the previous chapter.

The evaluation has been classified into two sections. In the first section, presentation of some existing tools and frameworks will be done that can be used for application realization. In the second section of this chapter, we will present and evaluate different technologies that can be used to create Farmers Dashboard of the selected framework and tool.

4.1 Existing dashboard frameworks

There are many specialized frameworks and technologies that offer a solution to create a custom-made solution for a dashboard. Out of many three of them have been selected to be evaluated in this research.

During this evaluation, a common goal is to reduce the development

(43)

time, increase re-usability, provide the extensible dashboard and fulfilling the requirement of end-user that is necessary to realize an application idea.

4.1.1 Knowage

Knowage suite [28] is a professional open-source suite for modern business analytics and is free to use for anyone in the world who wants to take advantage of a very flexible and easy to use data visualisation suite, it has both a free and a commercial version. Knowage adopts open standards and can be used in various environments without con- siderable requirements. Moreover, its modules can be combined with each other to get a tailored solution and provides advanced reporting, analytics, data visualizations, data mining, multi-dimensional analysis and also provides advanced self-service capabilities that give autonomy to the end-user, here able to build his own analysis, explore and organ- ize his own data space.

Knowage is a smart solution that is composed of several modules, each one conceived for a specific analytical domain. They can be used individually or combined with one another to ensure full coverage of user requirements, allowing to build a tailored product.

Currently, the preferred solution by DEMETER is Knowage suite which has been used to implement the dashboards in DEMETER. The very flexible and dynamic software module has made it possible to deliver an adaptive framework that allowed the porting of old DSS before DEMETER project into a concept about visualisation.

The Knowage adaptability has been amply demonstrated during the development phases of each individual dashboard as it allowed the technical coverage of all the specifications introduced by the area and component leaders. The use of different graphical widgets, such as tables, charts, or custom widgets have made up for the many requests and solved some problems that would occur using non-adaptive technologies. However, even if the technological framework used has responded very positively to the demand for DSS visualisation in DEMETER, satisfied heterogeneous requests, the path of technological maturity is constantly evolving and foresees areas for improvement,

(44)

also supporting particularly heterogeneous data formats.

In this complex world of software technologies and products, DEMETER wants to provide a better view and outlook over the farm activities and the farmer’s cooperation with both public and private actors, more efficient use of digital tools that the farmer applies and better and more customized decision support for each farmer, including decision support based on AI Machine learning from sensor data.

Despite being a preferred solution by DEMETER but it required a separate technical effort for creating a user interface again for each module from the APIs and for creating a dashboard for each farmer due to its dependency on knowage so it will be a bit critical to invite the all app providers to integrate their dashboards in our Farmers Dashboard Toolkit.

4.1.2 Grafana

Grafana [4] is a general-purpose dashboard and graph composer. It is focused on providing rich ways to visualize time series metrics, mainly through graphs but supports other ways to visualize data through a pluggable panel architecture. It currently has rich support for many databases like Graphite, InfluxDB and OpenTSDB. And also provide supports for other data sources via plugins. Grafana is an open-source tool in the Monitoring Tools category of a tech that is reportedly used by 1470 companies in their tech stacks, including Uber, Robinhood, and Nubank [24].

Grafana takes a unique approach of unifying the data but not the databases no matter where the data lives. It will visualize, translate and transform the data into a flexible and versatile dashboard and with advanced querying and transformation capabilities, it can customize panels to create visualizations that are actually helpful for the end user [24].

Grafana needs integration with data and creates its own interfaces and visuals. It will not give an opportunity to reuse the existing user interfaces and does not support the cross platform integration.

(45)

4.1.3 Micro-frontend (Bit.dev)

In recent years software development practices have been changing rapidly in order to enhance the reusability and efficiency of the application. Micro-frontend is one of the most important and emerging system architecture that has the potential to enhance developer productivity in building complex and scalable applications [38].

Micro-frontend architecture is a design approach in which a frontend app is decomposed into individual, semi-independent “micro- apps” working loosely together. The micro-frontend concept is vaguely inspired by and named after microservices [32].

Micro-frontends divide the application into vertical slices. Each slice is built from the database to the user interface and run by a dedicated team. This approach is related to the microservices architecture. But the main difference is that the service also includes its user interface. Here are the three main reasons why companies adopt a micro-frontends architecture:

Optimize for feature development – A team includes all skills needed to develop a feature. No coordination between separate frontend and backend teams is required.

Make frontend upgrades easier – Each team owns its complete stack from frontend to database. Teams can decide to update or switch their frontend technology independently.

Increase customer focus – Every team ships its features directly to the customer. No pure API teams or operation teams exist [23].

As micro-frontend decompose the frontend into different features as an independent module which is owned by independent teams such as each team has a separate area of business that the team is proficient in. Several industries like DAZN, Ikea, New Relic, SAP and others have adopted micro-frontends [33].

Micro-frontends are independent and can be integrated into any application. There are various tools and frameworks that are used to develop the micro-frontend. Recently, increasingly more micro- frontend frameworks have reached a sufficient level of maturity

(46)

that eases the adoption into new application areas. So during this assessment level, Bit.dev micro-frontend framework will be considered due to its popularity, completeness and maturity level which make it ready for a production solution.

4.1.4 Results and analysis

Requirements mentioned in chapter 3 has been evaluated for all three considered frameworks in table 4.1 that can be used to create the Farmers Dashboard Toolkit. During the evaluation, it has been accessed that Grafana and Knowage both can be used only if the farmers (end-user) requirements has been considered which is not suitable and will not help to achieve the goal of Farmers Dashboard Toolkit even though Knowage is previously used by DEMETER ecosystem. Both of them requires the dependency of developers to create dashboard modules or components from the third-party services for each farmer and it is not fulfilling the requirements of the app providers so Bit.dev micro-frontend framework is the one that can be used to develop this user interface toolkit which will fulfil not only the farmers requirement but also the app providers requirements.

4.1.4.1 Assessment metrics

During this assessment we will be using 0, 1, 2 evaluation scheme 0 = not fulfilled

1 = partial implemented 2 = fully implemented

During this assessment we have seen that bit.dev micro-frontend is the most powerful framework among all by fulfilling the maximum number of points as compared to grafana and knowage but still integration of micro-frontend is not simple and easy so we need to perform the second level of evaluation among other micro-frontend frameworks to see if there is some other micro-frontend framework to fulfill our requirements

(47)

Requirements Grafana Knowage Micro-frontend (Bit.dev) Increase use of

own data flow

2 2 2

Dashboard decision support

2 2 2

Overview and comparison with industry

2 2 2

Prediction for planning purposes

1 2 2

Net Economic benefit

2 2 2

Configurable dashboard

0 0 2

Cross platform integration

0 0 2

Reuse of existing user interfaces

0 0 2

Decoupling of application

2 2 2

Scalable dashboard

2 2 2

Cost benefits 1 1 2

Development time

1 1 2

Simplicity 0 0 0

Single login 2 2 2

Total 17 18 26

Table 4.1: Assessment of existing frameworks

(48)

4.1.5 Micro-frontend frameworks assessment

In the previous section, it has been evaluated that the micro-frontend framework Bit.dev will be the technology with the maximum amount of points against our requirements and can be used for creating the Farmers Dashboard Toolkit. But the problem is that the integration of micro-frontends are pretty complicated in Bit.dev and it will create a problem when this platform will be extended to a larger level so there are other micro-frontend frameworks in the technology world.

So in this section second level of evaluation will be conducted to evaluate that which micro-frontend framework will be the best choice for Farmers Dashboard Toolkit to achieve the maximum goals and requirements. The three most popular micro-frontend are considered in this evaluation i.e. Bit.dev, Piral and web components.

4.1.5.1 Bit.dev

Bit.dev is an open-source tool for collaborating on isolated components across projects and repositories. The Bit.dev can be used to distribute discrete components from a design library or a project into a standalone reusable package and utilize it across applications. The Bit.dev facilitates the process of collaborating on U I components. Team members can share, maintain, and synchronize isolated components from different projects [5].

Bit.dev is a product-ready solution for building micro-frontends, which allows to create and manage frontends through independent components and it can not only build but also integrate components.

It leans toward the build-time approach for composing frontends, allowing developers dual benefits, robustness and safety of monoliths, as well as scalability of micro-frontends. At the same time, their cloud platform facilitates collaboration among teams to integrate these components in the end [31].

Bit.dev leverages component re-usability by providing an ecosystem for sharing components between the applications and across repositories. Components are the building blocks of modern web architec- tures. Encapsulated and reusable components with focused and well- defined APIs let developers build more robust software applications

(49)

Figure 4.1: Bit.dev micro-frontend framework [31]

more quickly. Bit.dev adds a semantic layer on top of repositories that maps files into components. This extra layer provides robust capabilities in making the component reusable across projects [6].

The individual components in this portion have been developed in a decoupled codebase and then shared and published on Bit.dev. This approach makes the components easily discoverable for being integrated with other components developed by other teams. Anyone using the Bit.dev framework has access to a similar workflow where teams can build, version, test, and publish each independent component. These components can then be exposed to the teams for collaboration and integration [31].

(50)

4.1.5.2 Piral

The Piral ecosystem offers the framework for building micro-frontend solutions, which allows the creation of distributed web applications reflecting the flexibility and modularized structure of a microservice backend [7]. Piral is the go-to tool for anyone looking to use Micro- Frontends to build portal applications. It enables the creation of a modular frontend portal/ dashboard application that is extended at run time with decoupled modules. It is the ready-to-use tool to build portal applications using micro-frontends. The decoupled modules are known as "Pilets", which is developed independently and ship with all necessary assets for micro-frontend development, including the code [31]. Before choosing Piral following points should be considered that it has a clear tendency towards React [19].

A Piral instance brings the overall design of the application and includes shared components that can be used by pilets. It also defines how pilets are loaded and where pilets can integrate their components.

The pilets i.e. feature modules bring the content for the application and include their own assets and dedicated dependencies. When the pilets have reached a certain maturity level, a developer can publish them into the Pilet Feed Service. Via the Feed Service, available modules can be loaded into the local development environment so that developers and testers can validate how their new module integrates with other pilets. Figure 4.2 illustrates the setup and process for developing with Piral. The prerequisites are fairly minimal and the developer only needs your favorite editor, a terminal, an Internet browser and Node.js installed. The Piral instance which will be the application shell and the pilets which will be the feature modules can be executed and debugged in the emulator on the local development machine [7].

4.1.5.3 Web-component

web-componet is a very popular and trending suite of different technologies allowing you to create reusable custom elements with their functionality encapsulated away from the rest of your code and utilize them in your web apps [40]. It allow the creation of technology- agnostic frontends which can be loaded by the browser. Shadow DOM

(51)

Figure 4.2: Process of development with Piral [7]

reduces the risk of CSS collisions and also protects against global styles leaking in [33].

Web-componets give us a lot of flexibility and freedom. It allows to introduce a micro-frontend in any application anywhere [34]. Web- componets create pieces of frontend that can be imported into web applications in a very elegant way. Those pieces can be packaged into microservices together with the backend [42]. It not only provide cleaner implementation but supports code and style isolation and allows the separate deployment by separate teams. The best thing about the web-componet is that they support different and the latest frontend technologies and frameworks like React, Vue, Angular etc. for rendering theU I which will allow the different vendors to deploy their own app in their own way [30].

Web-componet consist of three main technologies which can be used together to create versatile custom elements with encapsulated functionality that can be reused wherever you like without fear of code collisions.

Custom elements: A set of JavaScript APIs that allow you to define custom elements and their behaviour, which can then be used as desired in your user interface.

Shadow DOM: A set of JavaScript APIs for attaching an encapsu-

(52)

lated "shadow" DOM tree to an element — which is rendered sep- arately from the main document DOM — and controlling associ- ated functionality. In this way, you can keep an element’s features private, so they can be scripted and styled without the fear of col- lision with other parts of the document.

HTML templates: The <template> and <slot> elements enable you to write markup templates that are not displayed in the rendered page. These can then be reused multiple times as the basis of a custom element’s structure [40].

4.1.5.4 Assessment matrix

During this assessment we will be using 0 , 1, 2 evaluation scheme 0 = not fulfilled

1 = partial implemented 2 = fully implemented

4.1.6 Conclusion

Table 4.2 evaluate all three considered micro-frontend frameworks against the requirements. Piral is the one with the least fulfilling the requirements and this framework missed one of the most important requirements of the cross-platform micro-frontend framework. Bit.dev and web-componet had a very close neck to neck comparison and during the detailed analysis for both of them, it was found that both of them have one advantage over the other. In Bit.dev, the micro- frontend is exported in the form of a node package so we can include the micro-frontend into the dashboard with its node module but in the web-componet we need to add a javascript patch for each component into our user interface platform and then we can include it into our dashboard. But the web-componet have an advantage over the Bit.dev that it is not dependent on any third-party ecosystem and support the native technologies that will give more freedom to the third-party app providers.

(53)

Requirements Bit.dev Piral web-componets Increase use of

own data flow

2 2 2

Dashboard decision support

2 2 2

Overview and comparison with industry

2 2 2

Prediction for planning purposes

2 2 2

Net economic benefit

2 2 2

Configurable dashboard

2 2 2

Cross platform integration

2 0 2

Reuse of existing user interfaces

2 1 2

Decoupling of application

2 2 2

Scalable dashboard

2 2 2

Cost benefits 2 1 2

Development time

2 2 2

Simplicity 0 0 2

Single login 2 2 2

Total 26 22 28

Table 4.2: Evaluation of micro-frontends

(54)

We had also tested the cross-platform micro-frontends in both platforms and found that in Bit.dev cross-platform micro-frontend integration in the dashboard is quite complex while with web- componets it is very simple and it gives support to all native technologies and frameworks.

So on the basis of all these evaluations and assessments it is concluded that web-componet technology will be chosen to create the Farmers Dashboard Toolkit. In next chapter, it will be presented that how web-componet will be used to create dashboard concepts and design architecture.

(55)

Farmers Dashboard Toolkit

Farmers Dashboard Toolkit

Adaptive and configurable dashboard for Big Data

Javaria Hazoor

Thesis submitted for the degree of

Master in Programming and System Architecture 30 credits

Department of Informatics

Faculty of mathematics and natural sciences

UNIVERSITY OF OSLO

Farmers Dashboard Toolkit

Adaptive and configurable dashboard for Big Data

Javaria Hazoor

Abstract

Acknowledgements

Contents

I Introduction and Background 1

II Requirements and Assessment of existing

framework and technologies 22

III Design and implementation 42

IV Evaluation and results 63

List of Figures

List of Tables

List of Abbreviations

Part I

Introduction and Background

Chapter 1

Introduction

1.1 Motivation

1.2 Research questions

1.3 Main objective

1.4 Thesis contribution

1.5 Research methodology

1.6 Work behind the project

1.7 Thesis structure

Chapter 2 Background

2.1 What is Big Data

2.1.1 Technology stack of Big Data

2.1.2 Big Data workflow

2.2 Big Data visualization

2.2.1 Conceptualizing the Big Data visualizing for understanding the future need

2.2.2 Open challenges in Big Data visualization

2.2.3 Challenges for the visualization of farmers and agricultural data flow

2.2.4 Role of DEMETER in the European agri-food sector

Part II

Requirements and Assessment of existing

framework and technologies

Chapter 3

Requirements for the

Farmers Dashboard Toolkit

3.1 Vision and approach for the Farmers Dashboard

3.2 Requirements for the Farmers Dashboard Toolkit

3.2.1 End-user requirements

3.2.2 Third-party app providers requirements

Chapter 4

Assessment and selection of existing frameworks and

technologies

4.1 Existing dashboard frameworks

4.1.1 Knowage

4.1.2 Grafana

4.1.3 Micro-frontend (Bit.dev)

4.1.4 Results and analysis

4.1.4.1 Assessment metrics

4.1.5 Micro-frontend frameworks assessment

4.1.5.1 Bit.dev

4.1.5.2 Piral

4.1.5.3 Web-component

4.1.5.4 Assessment matrix

4.1.6 Conclusion

Part III

Design and implementation