Abylon Map – Abylon https://www.abylon.hu Data Analytics, BI & Collaboration for modern enterprises - Abylon Consulting Mon, 28 Nov 2022 15:16:00 +0000 en-GB hourly 1 https://wordpress.org/?v=6.9.1 https://www.abylon.hu/wp-content/uploads/2020/04/cropped-abylon_favicon-32x32.jpg Abylon Map – Abylon https://www.abylon.hu 32 32 Abylon Consulting’s experience with railway BI and data projects https://www.abylon.hu/bi-hu/abylon-consultings-experience-with-railway-bi-and-data-projects/ Tue, 19 Jul 2022 12:26:41 +0000 https://www.abylon.hu/?p=11291 We are very pleased with our long-lasting successful cooperation with MÁV (Hungarian Railways). The know-how we gathered during these projects make Abylon Consulting one of the most experienced company in the field of railway BI, data management, and data analytics not only in Hungary but also in Europe.

The post Abylon Consulting’s experience with railway BI and data projects appeared first on Abylon.

]]>

Railways - a world of diverse, complex and challenging BI and data projects

Abylon Consulting and the Hungarian Railways

Hungarian railways have been a priority sector in the life of Abylon Consulting practically since the company was founded. We have been working together successfully for 5 years now and we hope to continue our professional cooperation well into the future.

The planning, organisation and monitoring of rail transport is a very complex process. The individual areas involved are themselves highly complex units, whether it is maintenance, sales or traffic management.

It is often difficult for an IT service provider, consultant or developer to adapt to the specificity of the rail sector. Beyond the physical reality, there is also a complex legal regulatory environment to adapt to.

On the other hand, it is this complexity, these specific needs, that also present an interesting challenge for BI companies dealing with large amount of diverse data like Abylon.

At Abylon Consulting, we pride ourselves on our many years of cross-disciplinary experience in rail IT. The results of our BI projects serve real business needs, help strategic decision making, and also help track day-to-day business operations. Understanding needs, data and customer satisfaction are a top priority in our projects.

Rail BI and data project delivered by Abylon

Below we would like to give you a taste of some of the areas where we have been working together with the Hungarian railways in recent years:

  • Passenger sales
  • Train traffic, events affecting train traffic
  • Passenger count/occupancy data
  • Infrastructure sales
  • Vehicle technical condition monitoring
  • Measurement of staff performance

These projects involved a total of 13 data models/data marketplaces operating in an on-premise environment among the Hungarian rail transport stakeholders, as well as a cloud pilot project.

Our solutions work in a Microsoft environment, using SQL Server, SSRS, SSAS, SSIS, C# and of course Excel, which is of paramount importance for the users. Projects are well structured, following the structure required by Abylon Rapid Analytics and our best practices.

MORE ON RAPID ANALYTICS

As data markets continue to expand, we are actively assisting our customers with self-service reporting, which is an additional Power BI Desktop model.

The data markets and data warehouses are served by the Abylon Rapid Platform, with several special features designed to precisely meet client needs. 

For most of our data models, we provide an additional value add in the form of our advanced map-based data visualization solution built on Power BI – Abylon MAP, which provides a map-based view of any metric.

MORE ON ADVANCED MAP VISUALIZATIONS

In future posts, we plan to go into more detail about some of the real business cases and the related projects, of which we are justifiably proud. But for now, here are a few brief summaries of some of the projects we’ve worked on. Why they were interesting or challenging and what we’ve learned from them?

Near real-time data datawarehousing

Short description:

Usually data warehouses are updated daily. Practically, these updates are performed during the nightly idle periods so that the daily business is not disrupted by the load involved. However, in the case of traffic and related events, it was essential to update the data warehouses more frequently, several times a day, in order to be able to check the current, real-time data throughout the day.

What we learned:

It proved to be an exciting task to implement, resolve and optimize the intraday loads without any performance degradation on other systems, both in case of SQL and tabular models. We needed to tap into technologies that only enterprise application suites provide. Using our own framework, we had to choose a complex partitioning solution that handles current data in daily partitions and recent data in monthly partitions. Read more about this project here: Near real-time data warehouse

JÉSTAT+ Route decomposition and display of ticket dat

Short description:

A breakdown of the route description of the tickets sold according to the infrastructure included in the published timetable. Difficulties have been encountered due to the textual nature of the route description and the fact that some sections cannot be clearly assigned to railway lines (e.g. the PDF timetable available on the MÁV website suggests that some sections may belong to multiple lines, etc…)

What we learned:

With a thorough understanding of business logic and experience in the field, using SQL and .NET code, the appropriate logical decomposition was achieved. Creating a tabular model capable of handling data (>10 years of total sales data) was also a big challenge that required a lot of additional research. Read more on this project in this post: Railway data map visualization.

START-TÁR Railway vehicle condition evaluation

Short description:

Determination of availability, uptime and productivity based on the activities and conditions of each passenger vehicle (railcars and multiple units). For example, not only major breakdowns affect availability, but also if a vehicle has been in maintenance (activity) for 6 hours, its availability is 75%, and so on.

Experience:

On the SQL side of the project, it was an exciting task to compare vehicle activities and states, assigning a rating indicator to the vehicle based on the parallel, simultaneous states and activities, according to a defined priority. An other interesting part of the project was the handling of idling and defining the rating between two activities/states.

An award-worthy solution

We are planning to participate in the international Awards for Excellence in Railway Publication, organised by the Union of Railways (UIC) with a recent BI solution we developed together with MÁV (Hungarian State Railways) which examines the impact of delay events on the rail network.

The solution is showing, through a user-friendly report, that a primary delay event can lead to additional so-called secondary delays elsewhere on the rail network, and thus cause a degradation in service quality.

A special business logic for the correlation between delays has been developed, which efficiently and almost synchronously collects the delay statistics caused by disruptions.

An example showing how one delay is effecting other lines, trains in the network.

Successful rail BI cooperation

We are very pleased with the many years of successful cooperation with MÁV. We are proud of the many successful projects and of the experience and know-how we have gained in the course of these projects. As a result we can safely say, that:

Abylon Consulting has one of the largest expertise in the field of railway data management, data analytics and business intelligence (BI) not only in Hungary but also in Europe.

Found this post interesting?

Subscribe to our newsletter to receive updates of similar post and news!

Linkedin Facebook Youtube

Fore more interesting articles and news follow our social media channels!

Other Popupar posts

The post Abylon Consulting’s experience with railway BI and data projects appeared first on Abylon.

]]> Railway data map visualizations https://www.abylon.hu/bi-hu/railway-data-map-visualizations/ Mon, 11 Oct 2021 15:29:12 +0000 https://www.abylon.hu/?p=9407 In the world of Power BI, there are no identical solutions. Both, the business requirements and the data are almost always different. In the ETL

The post Railway data map visualizations appeared first on Abylon.

]]>

In the world of Power BI, there are no identical solutions. Both, the business requirements and the data are almost always different. In the ETL processes as well as in data modeling, we need to find the solutions that can provide the answers to the questions in the simplest and fastest way.

The complexity of the data processing, the running time, the transparency of the tabular model, the complexity of the measures used and the running time are also very important metrics measuring the quality of our work.

When visualizing railway data on a map, the appropriate model can be identified knowing the business meaning of the data and its volume and complexity. Typically, there are two types of data available in case of line-based infrastructures:

  • Point
  • Route (Railway route

Connecting the fact data with the map objects

Point data can be an extraordinary event (eg accident), data related to an infrastructure element (maximum speed, condition, scheduled maintenance, cost) or detailed traffic data for a certain train based on its railml schedule and actual traffic data. This data can be fitted to a star schema, and the dimension table serving the map can be linked to the model with a simple One to Many connection

In the image below, the EU_OBJECTS dimension table is the display source of the map, it will display the objects (line or point) broken down by ID, based on their type. The EU_TRANSPORT table contains the factual data:

Note: Abylon MAP requires the usage of a unique ID and and object type, so it can be mapped to geojson files, or individual lines can be displayed separately by direction. Click on the button below for more details.

MORE ON ABYLON MAP

Organizing routing or interval type data into a model

Traffic data that contains a measured value along its entire route is called reference data. Tihis includes passenger traffic, ticket sales data, travel from A to B, or train data that do not change from the start point to the end (eg weight, railway company), but do not include traffic time data or values calculated from them.

We have several choices regarding this data. The problem is similar to period-type data, and there are three available options:

Interval data is broken down into the smallest point-like data required:

Like from time interval to days, and then from relations to line segments and stations. The days of the time interval can be extracted into separate rows in the fact table and these (as points) can be added into the data model.

Tabular On the fly:

We entrust the data model to sort out the data using one or two connection tables. See later.

DAX on-the fly:

When working with small amounts of data, instead of complicated modeling or significant data multiplication, we can filter using a DAX filter, in which case the dimension and fact tables are not linked within the model. Continuing with the time interval example: calculate ([something],filter(’Fact’,[datfrom]<=max(’date’[date]) && [datto]>=min(’date’[date]))

Combining solutions 1 and 2 to find a calculation optimum.

In this case the fact data rows are not broken down to an elementary point / section level, but to a higher level (e.g. railway line) and the model itself remains in accordance with number 2. (author’s note: in my opinion, this is the best method, we can provide detailed, pin-level data with it, but more frequent queries tailored to business needs can occur faster).

We will now discuss solution number 2 in more detail.

Tables underlined in yellow color provide the dimensional expansion:

The row of the fact table contains data for complete routes, to which is attached the routes M2M table, which contains the unique values of the route identifiers.

Routes are connected with a many-to-many relationships to the MAP object level, whose final dimension table contains the map data: coordinates, names, and so on.

The Fact data is like this:

ID    

FROM  

TO  

DIRECTION_ID  

123  

A

D

23

124

K

O

24

The directions M2M connection:

DIRECTION_ID

23

24

The MAP technical table is for extracting the directions:

DIRECTION_ID

MAP_OBJECT_ID

123

A

123

B

123

C

123

D

124

 

124

 

124

 

This approach allows you to assign data to an elementary map object without multiplying rows.

So, if we filter for an individual object in the MAP’s dimension (e.g an elementary section of line), the engine will calculate all of directions that have this element in the route description, and the fact data will be filtered for them

Which approach is the best?

In the case of point data, the question seems clear, although we have already seen a case where a set of factual data was related to a railway station and also to the related section. With railway data models, it is particularly interesting that you need to handle the relation to a railway line itself, so the master data of the map should contain the stations that belong to several lines multiple times.

In terms of where to start with relative/related data, the quantity and quality of the data is the decisive factor. With that knowledge (and clear business requirements) an experienced data modeler can easily chose the right approach.

If we set aside the railroad example and look for a more straightforward example, successful management of the time interval may still be the right example

Business Case

Solution

Short time intervals (1-3 days)

It is recommended to transform the time interval to a point regardless of the amount of data, as it does not cause an increase in the size of the rows of factual data.

Long time intervals with limited data (such as contract records)

In this case, the model-based on-the-fly and the daily breakdown would result in unnecessary duplication of rows, but the the dax on-the-fly approach is a good choice and we can treat the factual data with interval-filtered measures based on an independent date table.

Medium / variable time intervals, large amount of data (hotel reservations)

If our calculations show that breaking down the data into days would increase the number of rows by an order of magnitude (minimum 5-10 day average time interval), it is recommend to use the model-level on-the-fly approach.

Medium / variable time intervals, with extremely large amount of data or special requirements

In some cases, it is necessary to try to keep the size of the tables implementing M2M connections manageable. In this case it may be worth combining the different solutions:

  • the lines are broken down into weeks
  • we create explanatory tables for the weeks and for the combinations within a week.

In this case, there will be 28 rows for a week, which can provide a favorable breakdown even for a multi-year interval.

Custom Power BI map visualizations for railway companies

Abylon MAP is built on top of Power BI offering, custom, advanced map visualization features for companies dealing with line-based infrastructures. It is ideal for railway (or other transportation companies dealing with line-based infrastructures) looking for an affordable, customizable, easy-to-use map visualization tool for Power BI.

 

  • Infrastructure-specific Power BI map visualizations
  • Visualization of line-based networks using both circles and lines
  • Full integration with Power BI (built on top of it)
  • Active maps based on Power BI models or GeoJSO
MORE ON ABYLON MAP

The post Railway data map visualizations appeared first on Abylon.

]]>