Accelerate your digital transformation using Python

)In this article I will also share the thoughts of a handful experts, who were willing to be interviewed by Lisa de Kluijver and me and contributed with their experience.

A special thanks to:

• Lieke Melsen – Assistant Professor Computation Hydrology at Wageningen University
• Thijs Damsma – Data Engineer at Van Oord
• Jacco Hoekstra – Professor CNS/ATM (Communication, Surveillance, Navigation / Air Traffic Management), and former Dean of the faculty of Aerospace Engineering, at Technical University Delft
• Johan Tuls – Structural Engineer at BAM Infraconsult
• Philip Vollet – Senior Data Engineer at KPMG and Machine Learning and Natural Language Processing (NLP) Specialist
• Maaike van Putten – IT trainer and developer at Bright Boost

This is part of a series of articles about engineering and Python. Each time we will go deeper into engineering content and talk about solving specific problems. Other articles on this subject are:

• How are engineerings creating professional software tools?
• Parametric design in the cloud with Grasshopper
• Automating RFEM model generation with Python in a web application
• How to make an IDEA StatiCa Concrete model with Python
• Combine the power of Dynamo and Python in the cloud for easy sharing and accessibility
• How to integrate a PLAXIS model into a cloud-based parametric engineering tool with a few lines of Python code
• Automated Pile Driveability analysis with GRLWEAP and Python
• Using Python to create parametric SCIA models in the cloud
• An application with custom Python scripts to save time and reduce errors

Boosting your digital transformation

The frustration of the iteration

Eight years ago, I graduated from Aerospace Engineering at the University of Technology Delft. During my study I used all kinds of parametric design models, advanced analysis, and optimization techniques. When I started working as a Structural Engineer, I was disappointed: there was never time, budget, and willingness to create professional software tools that could make a big difference in the long term.

Frustrated by the large amount of boring and repetitive tasks of a typical engineering project, I went on making software tools myself, using project hours. This was a little risk-taking and it felt a bit clandestine. I was supposed to just update the models and reports, but it seemed that no one cared as long I was meeting the deadlines. I never regret this decision, as those tools enabled me to react more quickly to the many design changes in a project every time.

My Python toolbox

It took me a while before I discovered the best programming language to do this. At university, I learned to use Matlab; however, not every engineering company I worked for could afford those expensive licenses. Frustrated by the limitation of my Excel sheets I decided to give it a boost by learning Visual Basic (VBA). What a bad choice. I wish that someone had told me: use Python!

With time, I had a small library with codes. It was like having a box with digital building blocks that I could connect and combine to solve a specific problem, making it easier and quicker to make those programs each time. The advantages of having a software library were obvious, but the question remaind: Why were the companies for whom I was working not building their own libraries?

Going beyond the project system

It took me some time to understand. Many companies work project-based, and that was exactly the problem. Everything needs to be done between the scope, budget, and limitations of a project. This project-based model does not stimulate creating software as a company, that can later be used in several other projects as well. Due the lack of good software, design changes need to be done individually by each discipline, models are updated manually, and there is a lot of (slow) communication. Therefore, each change has a significant impact on the progress. I call this the sawtooth effect (see figure).

Building digital assets

Starting to build custom software tools as a company requires a change in mentality. It is investing upfront and benefiting later, which may seem risky. Making a proper software tool from scratch is risky as well and can require a huge effort at the beginning of the project. However, when done properly, progress can go exponentially and calculating and reporting a design change can become a matter of minutes (see figure).

At the beginning, progress is hard to track, which makes people nervous. I think this is a big reason why many companies are not taking the first step to creating custom software for their projects. The solution is therefore investing in digital building blocks. Once you have a few of those in your possesion, you can reutilize and connect them to quickly make software applications and benefit immediately. It is clear: Digital building blocks are assets for the company, that everyone can use, and provide a long-term benefit.

"Construction companies grow linearly, but programming their own building blocks makes it exponential. Programming building blocks is like building up capital. Each time you can make more, better, quicker and cheaper software applications, allowing you to make a big step forward." - Johan Tuls, BAM

Some background info about Python

Python was created by Guido van Rossum and first released in the beginning of the nineties. The aim was to help programmers write clear, logical code for both small and large-scale projects. That is exactly the reason Python has grown as big as it is now.

"Python is the future because it is a lingua franca. It transcends all platforms and domains. You can use it for infinitely different software applications. This makes Python unique." - Jacco Hoekstra, Technical University Delft

To sum it up, these are the most important reasons for why Python has become so popular:

• It is free and easy to learn! Out of all programming languages it is probably the one that resembles English most, meaning you can go from thought to code very fast and intuitive.
• It is not only about the language. It is also an amazing open and accessible world of knowledge and available resources.
• You can do anything with Python, from simple maths to scientific calculations on a supercomputer, including Data Science, Machine Learning, creating web-based applications, and much more.
• It is very scalable and suitable for working together with other people, even from different disciplines.

With Python, you can start small and slowly build up a digital empire. Do you not believe me? Google, Facebook, Instagram, Spotify, and Netflix; they all use Python as well! Just Google it ;).

Digital Building Blocks

I noticed that the common approach to build scalable software and cooperate with other specialists is literally thinking in digital building blocks that can be connected to each other. This make the process versatile and scalable.

Some benefits of using this building block approach are:

• You can build code fast by utilizing someone else's work! No starting from scratch.
• It enables people to share knowledge in the form of code!
• It makes it easier to replace parts of your code!
• It makes growing the code more controllable, as each block has a specific function!

This approach started in the software development community, with object-oriented programming and packages you could use to make your own software tools. Nowadays, it is becoming more popular in Engineering, Data Science, Architecture, and the Scientific Community as well.

Anything you can connect and command can be seen as a building block. It could be from a simple script, from one of the many free Python packages, or from FEM software.

Objects: The building blocks of Python

Python is object-oriented. This allows you to break down your software into small problems (we call them objects) that you an then solve seperately, one object at a time. It is like building something large out of smaller components. This is a good match since it is very similar to how I, as an engineer, look at physical objects that have to be designed, built and maintained.

Object creation

To create an object, you need to define a class. A class is a template for creating objects on which you are able to define its' behaviour and properties. A class can be used to make as many objects as you need. The great thing is that you can use properties of different objects for other calculations or for creating a ‘larger’ object as well. For example, you can create many beam objects, and then use their mass and stiffness properties to calculate the deflection in the bridge object.

Free information flow

Information between objects can flow very freely. This is totally different from scripts typically made when using Visual Basic, Matlab or FEM packages. Here, the code runs top-down, meaning that changing something in the code will affect everything beneath it. This is like playing Jenga. The more complex it gets, the greater the chances become that it will collapse. With Python however, this problem does not exist.

In general, you can see objects as digital building blocks for making a larger software program. Allowing different specialists to build, maintain and improve their objects, and share it with the rest of the world to make something even more awesome!

Free building blocks

The Python Package Index (PyPI) has 243,692 packages you can use for free! Most are not self-standing software, but building blocks for creating larger software tools. There are packages for visualizations, Machine Learning, Data Science, numerical calculus, and much more. Their quality is very high, so use them well and build your code fast!

For example:

• Numpy can be used to do all kind of matrix operations. Its functionalities are comparable to MATLAB.
• Pandas helps you manipulate and analyse large sets of data without having to learn a specialized data-processing language.
• PyTorch enables you to use machine learning without being a specialist.

Which ones do you use the most?

"There is a democratization of knowledge. It has become easier to use people’s tools and get insight into other disciplines thanks to the many available software packages." - Thijs Damsma, Van Oord

External software is just another block

Quite often I hear that engineers are facing the limits of the standard software packages they use and this is leading to cause a change in mentality. Nowadays, standard software packages are not considered to be the total solution anymore. These software packages are just part of a greater process and engineers are now searching for connectivity.

These days, a large number of external software packages can also be commanded with Python. This is very powerful, as it enables you to use them like any other digital building block in your code. For example, you can combine a cost model made in Excel with a structural calculation to find the cheapest solution that satisfies the structural requirements. This can also be done with other tools, such as Grasshopper, Dynamo, SCIA, Plaxis, Diana, Excel, ANSYS and Deltares D-Series.

What’s next?

Knowledge distribution

As an engineering community we still have a great challenge: the distribution of knowledge. What do you think is the best way of distributing digital building blocks and making knowledge available to others? I would like to hear your opinion about this topic, so don't hesitate to contact me!

Personally, I think there are two groups: the programmer, who can read, understand, and use a code; and the end-user who cannot program, but is going to use the tool.

For the first group, good distribution channels are still missing. As a result, many of these specialists are programming the same theories or creating programs for solving the same problems over and over again.

"We lose enormous amounts of time by programming the same models. If there was a central place where these models where available, we could invest that time in going forward." - Lieke Melsen, Wageningen University

Marketplaces for software applications

I believe soon there will be marketplaces where specialist can offer the software applications they made, so they can be used as digital building blocks for solving larger or more specific problems. There should be a financial compensation, such that companies and freelancers see this as an investment, and the creation of specific engineering tools is stimulated. If you are interested in this marketplace, let me know as well!

Additional challenges

For the second group, the ones that cannot read or write code, another solution must be found. I am a strong believer in user interfaces and online software applications.

“A big benefit of web-based applications is that the different disciplines have access to a common environment and always can make decisions based on the latest version of the information.” - Philip Vollet, KPMG

Want to know more?

This is the first of a series of articles about engineering and Python. We will talk about how you can make professional engineering applications that everyone can use, as well as on how you can make parametric design tools using Python and common engineering software like Grasshopper, Dynamo, SCIA, RFEM and IDEA. To write these articles we interviewed engineers that make these kinds of tools on a daily base. Real pros!

Stay tuned! Follow us on LinkedIn, so you don't miss anything.

Maybe you wonder why we are writing this. VIKTOR is on a mission to unleash the world’s real engineering potential. We are developing a platform where engineers can build their own web-based applications using Python. We work closely together with engineering, construction and manufacturing companies and help them boost their digital transformation. We would love to hear your feedback on our ideas, try out the platform, and help us create the perfect tools that enables you to:

AUTOMATE THE BORING. ENGINEER THE AWESOME!

If you are interested in developing your own parametric design tools, make sure to check out our solutions page.