Records in Elm are quite like JavaScript objects. In ES6, destructuring objects can produce compact and concise code. This article explores some techniques that Elm offers to the same effect.

What in ES6 is known as destructuring, is referred to as pattern matching within Elm. Pattern matching records appearing as function parameters is our focus for now. Yet, the technique is applicable in other cases as well. The example below presents the basic syntax. The code declares a type for a shopping cart item and a function that calculates the total for that cart item:

By changing calculateTotal's declaration somewhat, we extract shorthands for the values we are interested in. This declutters the calculation, only the important parts remain.

There might be cases where a record contains more fields than interest us. We can create shorthands for a subset of the fields while also retaining a reference to the whole of the record.

Here we unwrap unitPrice and amount to calculate the total, but keep a reference to the whole record. The otherFunction needs access to the rest of the record. It also receives the calculated total as an argument.

Let’s move on to an example from the real world. Records appear somewhere in the model of most elm apps. Different parts of an elm app uses the model in different ways. Some functions update the model and change only one or a few of the fields. Other functions, read larger numbers of members of the model at the same time. This happens often in view code.

Behold the Model of the Elm todomvc app.

todomvc's view function defines the top level structure of the view. Through varius calls, it delegates parts of the model for rendering:

We could apply pattern matching to avoid repeating model.:

Finally, it's worth to consider a caveat. Over-applying the technique obscures the origins of each field in the function body. This happens quicker as functions bodies grow. Still, pattern matching is a necessary and useful tool to grasp

Unpacking Records

The year 2020 has certainly been a special one, but that hasn't stopped Jetbrains from continuing its work on the Kotlin language and its ecosystem. In March we got the 1.3.70 version, an incremental release, which included some new functions and classes in the standard library. And after the summer break we got the all new 1.4.0 version, a feature release, which came with some really sought after changes to the language and even more changes to the standard library.

1.3.70 release

This version introduces a new kotlin.collections.ArrayDeque, which is conceptually similar to java.util.ArrayDeque, but it has its own implementation. And as such it is now possible to use if you're targeting a Kotlin/JS or Kotlin/Native.

Some other neat additions are the builder functions for common collections; buildList, buildSet and buildMap. And of course you can do whatever you want within the lambda, so this makes it really easy to conditionally add elements to our collections.

As if that wasn't enough this release also introduces scan and scanReduce. Functions who are closely related to fold and reduce, but returns the whole sequence of intermediate results instead of just the final result.

This is by no means an exhaustive list of changes, and we recommend taking a closer look at the changelog

1.4.0 release

This release brought a number of changes to the language, tooling and even more changes to the standard library.

SAM (Single Abstract Method) conversions

SAM (Single Abstract Method) conversions for Kotlin interfaces are now supported. Previously SAM conversions were only possible for interfaces defined in Java, but with the new fun interface syntax it is now possible to create similar constructs in kotlin. As the example show we are now able to call accept on our Predicates, this is possible since behind the scenes the kotlin compiler actually creates classes and instances that implement the abstract function.

Mixed named and positional arguments

When using named arguments in kotlin 1.3 you were forced place all positional arguments before the named ones, e.g you could do quadraticEquation(5, 6, c = 1) but not quadraticEquation(5, b = 6, 1). In kotlin 1.4 this restriction is relaxed, you you may mix positional and named arguments as you please as long as the arguments are in the correct order.

This can be especially helpful if you have a function with a couple of boolean flags;

Trailing comma

Kotlin 1.4 also adds a possibly contentious new feature with its trailing commas, e.g listOf("A", "B", "C",). The feature affects arguments and parameter lists, when entries and components of destructuring declarations.

New compiler

The new compiler was available behind a feature-flag in version 1.3, but is now the new default. With it comes a new type inference algorithm which is even better than the previous version at inferring types, smart-casting and SAM conversions.

Additions to the standard library

Version 1.4 continues to add even more stuff into the kotlin.collections package;

setOfNotNull() is added, mirroring its sibling listOfNotNull

shuffled() is added to Sequence, whereas it was previously only available on Iterable

min() and max() has gotten a deprecation notice, guiding you to use minOrNull() and maxOrNull()

The arrays have gotten some love as well, introducing shuffle(), onEach(), reverse() and more.

The first stable version of kotlinx.serialization is shipped - version 1.0.0-RC. Is this the end of weird serialization bugs in kotlin? We'll see.

This was just a quick overview of what I personally found interesting in the changelog, and a lot of things were left out. I encourage everytone to take a look the full changelog for a more comprehensive list of changes.

1.5.0 and future roadmap

The Kotlin roadmap outlines a few areas where we can expect changes, including further work on the new compiler and extra improvements in tooling and multiplatform support.

Perhaps we'll see support for functions with multiple receivers (KT-42435), JVM sealed classes (KT-42433) and JVM records (KT-42430). Or perhaps we'll get Kotlin/WASM (KT-42292), only time will tell.

The next feature release is scheduled for March 2021, and we'll be eagerly await its arrival.

Kotlin changelog

His palms are sweaty, knees weak, arms are heavy There’s vomit on his sweater already, mom’s spaghetti He’s nervous, but on the surface he looks calm and ready To drop bombs, but he keeps on forgettin’

// Eminem, “Lose yourself”

As often - a rap artist articulates it the best. How often have you heard that the average persons biggest fear is to stand in front of others and present something?

This probably stems from The Book of Lists, which in 1977 reproduced a 1973 survey by Bruskin Associates. Here’s the top ranking of the worst human fears: 1. Speaking before a group 2. Heights 3. Insects and Bugs

The fear of death itself was at a humble seventh place. So, if you feel like this, you are definitely not alone. However, just the insight that this is a completely normal response to stage fright, may not be extremely helpful when you are in a public speaking situation. Doing a simple web search will provide you with a ton of articles with different thoughts about how you could cope with what is often percieved a very stressful situation. Here is an outline of some things that have helped me, as a fairly introvert developer, in the past.

Remember that the presentation is (rarely) about yourself

Usually you are there to present a topic you know well and care about - and the audience is to there to listen to the topic, not there to judge you.

No one is there to see you fail.

Mostly, people will like you to succeed. It will be the most comfortable for everyone in the room, and the audience wants to get something out of your presentation.

Breathe with your stomach.

If you haven’t conciously tried this, it is highly recommende in a situation where you are under pressure. If you are like me, you will feel that you have much more control over your surroundings.

Bring your motivation into the presentation.

This may sound a bit fluffy, but hopefully, there are at least a few things you really would like to tell the audience with your presentation. It could be a couple of key sentences on why to use a certain technology, or a valuable experience you have had, for instance. Focusing on getting to those few things works for me - the fear of stumbling on your words more or less disappears into the background.

In the next chapter I will go into more depth about my personal experiences in this realm.

How to cope with stage fright

We thought we were ready. We finished our degrees, had done quite a few design projects, and read books about UX. Little did we know that there are many aspects of being a UX designer that our degrees did not prepare us for. As we now approach our four-month mark working at Bekk, we have reflected on five aspects of working as UX designers we wish we had known about before starting. Maybe they can help you as an aspiring designer?

1. Understanding business and politics is just as important as understanding users

It might sound a bit strange, but when working as a UX designer, everything doesn’t revolve around the needs of the users. We know. Mind-blowing, right? The needs of the client are often equally as important – whether they need to make money, have a social mission, or need to implement changes due to new laws and regulations. This means that we, as designers, might have to think “how can we, with these business needs, design for good user experiences?”.

When we started working as designers, we noticed how our colleagues were talking about politics. We eventually understood this referred to the power dynamics between various stakeholders in the organization in which you work. Figuring out what responsibilities these stakeholders have, how their success is measured, and what engages them is an important part of understanding the needs of the client. Understanding the stakeholders will help you gain a position in which you can execute your design. If you’re not in the position to execute your design, making an impact can be difficult – regardless of knowledge or skill.

“But how do I prepare for this?”, you may ask. Our best tip is simply to be aware of this insight when you start working. Look out for the different stakeholders, their needs, decision-making powers, what defines their success, and what engages them. Then, as with many things in life, you learn as you go!



2. You can improve a design after its release by continuously gathering insight

In the traditional design process that we were taught, a designer figures out how the product should work, perhaps by conducting interviews and surveys. Then he or she starts designing a prototype to hand over to the developers. One shortcoming of ending the design process after design handoff is that many insights await after a product is launched. Continuously learning and adjusting a design is called continuous design. Here are three ways of working with continuous design that we've tried out:

Observe and measure real behavior with Hotjar

Hotjar is a tool that can be integrated into your product to learn more about users' behavior. Among other things, you can create heatmaps of where users click, scroll or move their cursor to learn which functionality is used the most, find out what piece of information is the most popular, and so on. This can help you design for better user experiences by prioritizing which information to show in a particular place. Hotjar also records how users navigate your product, making you able to see how real users interact with the product.

Ask questions about changes you’ve made

Getting feedback on ideas and gathering insights is essential when designing. That’s probably why conducting interviews or surveys are popular ways to find out if you’re on the right track. We encourage you to continue questioning users also after the initial design is released. By doing this you can receive feedback on how users reacted to the change and how it affected their task. This can lead you to continuously making adjustments to your design.

Consider creating fake doors

You can create a “fake door” by implementing a button or a tiny bit of functionality to gain the users' attention. When the user clicks the button it is explained how the functionality isn’t done yet. They are also presented with a small questionnaire asking about what they would like to see or which functionality that would be relevant to them. This may result in feedback on the concept before it’s fully implemented.



3. The many benefits of including developers in your design process*

At school, we often worked in groups of designers. This is a fun and educational form of learning. What we found through our recent months of working is that including developers can also yield several benefits that you should be aware of.

Involving developers in your design process can provide the technical insight needed to make sure your designs are technically feasible. As team members, they are also invested in the product and can provide valuable feedback. When developers are included in your design research process, they can be your allies and share an understanding of why certain features are more important than others. This is of course a vital part of working in a team. You can also find out what each developer is interested in. Maybe one of your developer colleagues wants to learn how to make animations or another would like to focus on software architecture. Knowing this allows you to tailor tasks according to your team and thus working with their strengths and interests.



4. Sketches are a valuable tool for communication

Sketches are simple drawings, usually drawn by hand, used to explore product ideas in the early stages of the design process. They are known for being useful because they are quick and easy to make and communicates your idea without costing too much time and effort. While this is very true, we find the real value of sketches to be as a tool for communication. When talking with various stakeholders that are part of the product development process it is easy to assume that there exists a mutual understanding. However, as we designers know, everyone can have different mental models. Having rough sketches in front of everyone or making them on-the-fly in the meeting (sounds intimidating, but is so useful) ensures that everyone is more likely to be on the same page. If a developer is present too, it can help uncover unknown technical complexities that you were unaware of. The sketches can then be used to talk about technical complications and how you might solve or work around them (yet another example of why developers should be included in the design process).



5. Writing lists will help you keep on top of your work

A short and concise tip from us and many other designers in Bekk is to write lists! As we're sure you can gather from everything we have written about until now, a designer's role is multi-faceted and includes staying on top of many things at once. Having a way to keep track of decisions being made, tasks to do, people to talk to, and meetings to attend will not only help you remember everything you have to do, but will also show you everything you have achieved! Sometimes, if you have had a day full of meetings, it can feel as if you haven’t done much work. By looking at your list you can see all the meetings you have attended, people you have talked to, and decisions that have been made: all important aspects of the work.



____________________



These are five different topics that we have learned a lot about during our first four months as UX-designers. Had we known this before, starting our careers might have been a tad easier. But what’s life without a challenge, eh?

* We also believe that including business developers in your design process is highly valuable, but that's a topic for another time.

Five things we wish we knew about working as a UX designer before we started

I'm part of a group that holds introductory courses on web development for students and new employees in Bekk. In the course we say that there are three ways to retrieve data in JavaScript, but we only explain one of the methods. Now, let's see how each of them works: callbacks, promises and async / await.

In the spirit of Christmas, we'll use a suitable analogy. Let's say Santa requests that an elf fetches a gift for a child on his list. He does not know when the elf will be back, but when it does he wants to put the gift into his bag of toys. This is 2020, after all, so of course he uses JavaScript to complete the task :santa:

Our elf in this analogy will be represented by the function getGiftForChild(), which we will define for each of the three methods. The elf will retrieve the gifts using the magical url https://santas-gift-storage.northpole/gifts/nameOfChild, which returns a gift for a given child's name. We also have access to the global array bagOfToys, which is to be filled up with Christmas presents before Santa is on his way.

Callbacks

The first way to fetch data utilises that we can pass a reference to a function A as an argument to another function B, which then is going to call A at some point. Here, A is called the callback-function, while B is the higher order-function. This is a very common pattern in JavaScript, so it's likely you've seen it before – in event handlers, timeouts or even in the map, filter and reduce list-functions.

There are multiple ways to fetch data, but we're going to look at the classic, well-used API XMLHttpRequest. If you're not familiar with it, don't worry – the details aren't too important. We're interested in the way we use our callback, and I'll explain it below.

First, let's define the main function fetchChristmasGift(), which Santa can use to send an elf on his way with a child's name.

Then, we implement the gift retrieval using an XMLHttpRequest:

The event onreadystatechange is triggered when the state of the request changes. We assign a function to this event, which will be called when it occurs. If the state has changed to 4 = DONE, we want to do something: we want to call a callback function!

So let's create a callback function, to decide what happens when we have fetched our gift. Our instructions were to add it to Santa's bag of gifts, so we'll do that. While we're at it, let's include a check to see if we actually got the gift:

In order for the elf to know what to do with the gift, he must be sent some instructions. Therefore, we must pass the callback function to the higher order function getGiftForChild(). Let's take a look at the final result:

And that's it. We used a callback to do something with our data after we successfully fetched it.

But callbacks can easily get messy. There is a reason why there is something called callback hell :fire: (just Google it and see), and why this is not the preferred way of doing things. Let's move on to a new type of object that has another take on the matter.

Promises

The second way to fetch data uses Promises to handle asynchronous calls. Promises are objects that make asynchronicity a bit more tidy. For this purpose we use a function called fetch() from the Fetch API. Unlike the previous method, the responses are wrapped in Promises, so you don't have to explicitly handle all the nitty-gritty details anymore!

We start by defining the elf function getGiftForChild() which takes a child's name as input and fetches the gift from our URL using fetch(). This function will return a response in the form of a Promise.

Then, we can use the Promise prototype methods then() and catch() to do something when the Promise either resolves :white_check_mark: or is rejected :x:. Let's define the function fetchChristmasGift() like before, and call getGiftForChild() to fetch our gift:

In fact, this is also a way of using callbacks. We define what is to happen when we get a response, passing in a function to both then() and catch(). With Promises it's just being handled in a smoother way than our first example. But can it get even better? Keep on reading!

Async / Await

The third way to fetch data builds on Promises, but is generally a more elegant way of handling them that looks a lot more like synchronous programming. Assume we still have our function getGiftForChild() which returns a Promise:

This request can either fetch a gift successfully, or fail trying. Instead of using then() and catch() to handle these two cases, we can define an asynchronous function.

Here, we make use of the two magical keywords async and await, to explicitly wait for the Promise to resolve before we move on, and say that we do so.

In our analogy, it would look something like this:

We wait until we've retrieved a gift successfully, and then we'll add it to the bag :gift: To be sure, we can wrap it in a try / catch block to handle if the Promise is rejected:

We have to declare fetchChristmasGift() asynchronous with async, since it's using await within. That way, it's clear that something is being awaited where the async function fetchChristmasGift() is called. We can decide if we want to await that function as well, or go on with our business as usual in parallel. After all, the North Pole is a busy place.

And that's about it! I hope you learned something new about fetching data from this quick introduction, whether you're new to JavaScript or a veteran who has long since forgotten why you write code the way you're used to. Hopefully, Santa read this too and was able to find your gift in time for Christmas Eve :christmas_tree:

The Three Ways of Fetching

Maskinlæring er ikke lenger bare en hype, men i ferd med å bli etablert, velprøvd og kraftig teknologi som utnytter potensialet i store datamengder.

Samtidig sliter mange selskaper med å forstå og utnytte det strategiske mulighetsrommet for maskinlæring. De siste årene har vi sett utallige halvhjertede og irriterende chat-roboter som skal kunne «svare på alt», såkalt personlig tilpasning og tjenester som forsøker å gjette brukerens neste handling. Grunnen til at disse forsøkene ofte feiler er mangelen på gode data i store nok mengder, og mange selskaper synes å allerede ha gitt opp maskinlæring i tjenestene sine.

Hvorfor ikke bruke maskinlæring der så og si alle bedrifter allerede harstore datamengder - nemlig internt? I denne artikkelen slår vi et slag for å bruke maskinlæringsmetoder internt i bedriften for å effektivisere og forbedre tjenester, prosesser og bunnlinje.

Vi trekker frem noen områder der de aller fleste bedrifter vil ha potensiale for å ta i bruk maskinlæring.

Kundesegmentering

Mange selskaper opererer fortsatt med tradisjonelle, utdaterte kundesegmenter som «mann i 50-årene» eller «ung, urban kvinne». Planlegging og utvikling av produkter, tjenester, markedsføring og tilhørende ressursbruk skjer basert på disse oppdiktede segmentene. Hva med å la maskinlæring gjøre jobben, og finne mønstre og fellestrekk hos kundene du selv ikke visste fantes, og plassere disse i segmenter?

Prosessen kan være helt anonymisert, og man kan bruke data man allerede sitter på i dag som alder, bosted, kjønn, kjøps- og brukshistorikk, sivilstatus og annet. Om ønskelig og hensiktsmessig kan databasen suppleres med tilegnede eller kjøpte data om inntekt, boforhold, stilling, arbeidsgiver og annen data man får tak i. En enkel clustering-algoritme vil, forhåpentligvis, finne gode, uante mønstre i kundedataene, og plassere disse i passende grupper.

Disse kundesegmentene vil kunne være mye mer treffsikre og utfyllende, slik at bedriften kan fokusere energi og ressurser på å bygge de riktige verdiforslagene til det riktige segmentet. Kanskje ender du opp med noen av de samme segmentene du hadde gjettet på forhånd - men da har du i hvert fall lært mye mer om dem på veien.

Bemanning og distribusjon

Et annet område mange selskaper kan ha behov for effektivisering er innen planlegging av bemanning, logistikk og distribusjon. Antall kundebehandlere på jobb bør svare til pågangen på kundesenteret, varebiler bør være fullest mulig og kjøre færrest mulig kilometer.

Dette er et annet, viktig område der maskinlæring kan være virkningsfullt. Ved å bruke historiske data om kundeatferd - som antall henvendelser til support eller bestilling av matvarer - kan maskinlæring forsøke å predikere hvordan pågangen vil være i fremtiden. Der det kan være vanskelig å forutse handlingen til én enkelt kunde, så vil prediksjon på kundebasen som helhet kunne være mye mer treffsikkert. Ved hjelp av relativt enkle metoder som regresjon kan man bruke data om historisk atferd hos kundemassen, sammenstilt med eksterne data som værforhold, helligdager eller annet, til å predikere hvordan pågangen vil bli i kveld, i morgen eller neste uke. Dette vil være kraftfull beslutningsstøtte til planlegging av bemanning, logistikk og distribusjon.

Det vil vært både lettere, mer treffsikkert og verdifullt for Foodora å forutse hvor mange bestillinger som vil komme i løpet av en kveld, og hvor mange bud de da vil trenge, enn å forsøke å gjette hva hver enkelt kunde kommer til å bestille til middag og når.

En kombinasjon

Ekstra virkningsfullt kan det være om man bruker disse to områdene sammen:kan vi knytte kundesegmenter til en spesiell atferd, og forsøke å spå denne?

Lånekassen gjorde i 2017 et forsøk på bruke maskinlæring til å plukke ut studenter til såkalt bokontroll. De gjorde en segmentering på lånetakerne basert på hvem som hadde løyet om boforholdet, og dermed ikke hadde rett på stipendet de hadde fått. En clusteringalgoritme forsøkte å finne mønstre i dataene blant studenter som løy, og de som ikke gjorde det.

Utfallet av segmenteringen ble så brukt til å forutse hvem som kom til å jukse blant neste års låntakere. De gjorde et forsøk: ett utvalg for kontroll var tilfeldig, altså slik som det vanligvis ble gjort, mens det andre utvalget ble plukket ut fra clusteringalgoritmen.

Resultatet ble at utvalget der det ble brukt maskinlæring var dobbelt så effektiv til å forutse hvem som løy om boforholdet. Denne metodikken sparer både Lånekassen og hver enkelt lånemottaker for masse papirarbeid hvert eneste år - samtidig som det sparer Lånekassen (og staten) for titalls millioner kroner hvert eneste år. Det finnes en rekke slike eksempler på hvordan man kan knytte clusteringbasert kundesegmentering opp mot ulike typer brukeratferd.

Så..

Vi vil altså oppfordre selskaper til å ikke bare forsøke å dytte maskinlæring inn i brukeropplevelser der det egentlig ikke gir noen spesiell verdi for hverken kunde eller selskapet, men heller ta en kikk internt. Har dere noen store databaser som bare står og sørger for at systemer fungerer, men som kan sitte på uant, verdifull informasjon? Kanskje kan det være nettopp internt at maskinlæring faktisk kan gi stor verdi.

Hæ, maskinlæring nyttig for strategi?

8 developers, 1240 files, 102 665 lines of code. This is the story of how we translated our entire codebase from Flow to TypeScript and lived to tell the tale.

The why

Flows is ease of use with React / React Native, and gradual approach to typing made it an ideal framework to help us detect type errors and stay consistent.

I was not in the room when the decision to adapt Flow was made. But I recon the discussion went like this. How will this affect our current speed? Do we need to change the build pipeline? What new dependencies will we need to include? The answer is of course none, no, and babel-flow-plugin.

Fast forward two years and Flow was still with us, however it was getting harder and harder to ignore the elephant in the room. Flows momentum was not what it once was. Other Facebook products had made the switch to TypeScript, and in the community Facebooks commitment was being questioned. During November 2020 11 authors have pushed 44 commits to Flow’s master branch and merged 0 pull-requests, compared to 23 authors and 100 commits to TypeScript’s master branch with 85 merged pull requests.

Editor integration and developer experience with Flow left a lot to be desired. Our team is split 50/50 between IntelliJ IDEA and Visual Studio Code. Both editors has extensive support for autocomplete, type checking and refactoring for TypeScript, with Flow not so much.

Flow had done a pretty good job, but it was time to move on.

Getting ready to convert

In the repository there are three clients, web - app - widget. Code is grouped by domain and module, with platform specific implementations residing next to each other, read more in last years calendar. Up until that point we used custom file extensions to separate the web and native implementations. The native bundler would look for files with extensions .native.js and .js and the web bundler would look for .web.js and .js. A component folder would contain both a index.web.js and index.native.js and import of said component would only refer to the folder and let the bundler pick the correct implementation. This worked great and ensured only correct files where bundle for the platform. However, static code analyses, such as ESLint and Flow, would frequently get lost, and match against the wrong implementation. Resulting in a lot of false negatives and false positives.

It's easy to regard this as a limitation in the utilities, but actually it's symptoms of how much of a anti pattern the platform extensions is. It became apparent that it had to go. All index.native.js and index.web.js where renamed to native.js and web.js, and imports was updated with /native or /web. The result was rather nice. Instead of hiding the dual implementation complexity, it was highlighted, and static analyses worked as expected.

Another issue with any codebase of a given size is unused code. There is no point spending time translating code that nobody is using. ESLint can detect if a variable or method is unused within the file, which is great, however once you add a export in front of it your on your own. Finding unused code across the whole repository has been a combination of gut feeling and find in all files. The goal was to detect all unused exports as part of the testing process. Luckily this is not a new problem. There are several packages available. The code in js-unused-exports was relatively easy to understand, so I forked the hell out of it. Sure sex is great, but have you ever removed 812 unused exports.

Choosing a strategy

With the codebase pruned and in shipshape, there was time to choose a migration path. As with all migration there are really only two approaches; Big bang where nothing works until everything works, or hybrid where both systems exists side by side. It's really a question of where do you want to place the additional complexity and how long do you want to spend in the transition phase.

In a big bang approach there is only one type system, which gives you low complexity in build, but high complexity in the application. The transition phase is short, but intense, as all code needs to be rewritten before anything can be released.

In a hybrid approach there is two type systems, which gives you high complexity in build, but low complexity in the application. The transition phase is longer, with code remaining unchanged until you choose to convert it, but no block on releases.

Generally I prefer the big bang, just get done with it. But a hybrid approach seemed alluring. I decided to give it a try. The setup is quite easy. Building the application with both type systems is pretty straight forward. React Native supports it out of the box, for Webpack you need to enable the @babel/typescript preset for .ts and .tsx files using overrides. The hard part it getting the two type systems to talk to each other. There is no built in support for TypeScript in Flow or vice-versa. The only way for the two systems to understand each other is by manually bridging the gap using declaration files. Creating declaration files is not hard but tedious. Having to keep them in sync with the rapidly changing implementations, thats a recipe for disaster.

Lets say Flow imports a module implemented in TypeScript. That module takes one parameter, a string, and returns another string. This is also defined in the declaration file. Then one day we need to change the implementation so that it returns an object of strings, one for each supported language, instead. Usually you would change the implementation, run type check to see where it crashes, and update the usages. But now the TypeScript implementation differs from the Flow declaration file. Flow will still think it receives a string and calls all clear, regardless of what TypeScript says. You have now erected a iron curtain across your application, and every time you jump from one side to the other your risk of getting shot in the face.

This is a fundamental problem with the hybrid approach, and grows the longer you’re in the transition phase. After this realisation it was time for a U-turn, let's do a big bang instead.

Lets do what we came here to do

We have added support structures, polished, and made plans, but at this moment there are 0% TypeScript in the repository.\ We had blocked out three weeks to do the translation, full freeze, all hands on deck. I had one fear; cross cutting issues halting all work. When you have the luxury of full focus, it's important to utilise people as best as possible. A week before the work was scheduled to begun I created a parallel, secret (not really secret), master branch to prepare the work. The plan was come Monday, all code would have been converted, and all controls would be configured.

In order to achieve this I have to spend as much time possible fixing cross cutting issues, and as little time actually rewriting Flow code. Kahn Academy saved me an unbelievable amount of work with their flow-to-ts utility. It translates Flow to TypeScript keeping almost all of the type information, and renames the file to .ts or .tsx (depending on jsx usage). Translating the 1300 plus source files took under two minutes with only 10 needed a manual touch.

Once all files had been converted I could start work on getting control frameworks up and running, mainly ESLint. ESLint is built around a plugin architecture. It comes bundled with a standard javascript-parser and the built-in rule set, everything else is a plugin. Robert Cooper has written a great in depth on configuring ESLint with TypeScript. In our setup it was basically changing parser from babel-eslint to @typescript-eslint/parser, replacing flowtype/recommended with @typescript-eslint/recommended and @typescript-eslint/eslint-recommended extensions.

With ESLint up and running, it was time to fix formatting issues. flow-to-ts has built in support for Prettier, but it did not work with our ESLint - Prettier combo, and defaulted to standard formatting. The result was a lot of formatting errors. The solution was as easy as running eslint --fix which cleaned all issues, and made files look somewhat similar as before the translation.

Because the file extension had changed it was important that git marked the file as being moved, not removed + added, to preserve the history. Since Flow and TypeScript syntax is roughly equal, and ESLint ensured correct formatting, 99% of files preserved history.

The big freeze

The 3 week freeze was upon us. All that matters was getting the number of errors down to 0. There was no time for refactoring, and little to no code changes. By converting all source files to TypeScript we take on a lot of risk, and the last thing we needed was additional risk in the form of changes to the application. The outputted code should be close to identical to pre-TypeScript.

There was a lot of work to be done. Roughly 5000 TypeScript errors, 800 ESLint errors and 250 failing tests. To get one thing out of the way, and boost moral, we went all in on fixing ESLint errors. There was a lot of recurring errors "Unexpected any" and "Don't use `Object` as a type". None of these errors was hard to fix, but after a couple of hundred of them it starts to become quite dull. To my surprise, we reached 0 ESLint errors before the end of the first day.

With ESLint errors gone it was time to parallelise. When using TypeScript with Babel, type checking and transpiling are two separate tasks. Babel strips away the TypeScript semantics regardless of whether or not it works. Getting the applications to build was not dependent on passing type checking.

We defined three goals; Getting all tests to pass, getting the application to build, and passing the type check. Two developers focused on getting the tests to pass, two on fixing the build, and the remaining four worked on reducing TypeScript errors. At the end of the second day, the tests and build was fixed.

At the start of day three, with only TypeScript errors left, we needed some kind of moral boost. I hacked together a script to track our progress for every merge commit, and outputting it to a graph. Never underestimate the power of a graph. There is something deeply satisfying with comparing where we started and to where we are now.

How did it all pan out

We set out to use 15 days for a full Flow to TypeScript translation. The number of errors reached 0 after 8 days. Now two weeks later the number bugs related to the the translation is 2.\ Hats off to a great team!

War stories: The move from Flow to TypeScript

Hilde Marie Flesland Torall og Pia Solheim tar en prat med Camilla Brustad-Nilsen som er digital forretningsutvikler i Storebrand

Hvordan har Storebrand rigget seg for innovasjon?

NSM er bekymret for den samlede nasjonale avhengigheten av utenlandske skytjenesteleverandører. De mener Norge bør vurdere å etablere en offentlig sky på norsk jord, for å sikre nasjonal kontroll. Er dette en god idé?

NSM er i utgangspunktet positive til skytjenester, forutsatt at det blir gjennomført gode og riktige vurderinger i forkant. Det som nå bekymrer NSM mest er at virksomheter som er avgjørende for at samfunnet skal fungere, spesielt i krisesituasjoner, har gjort seg avhengige av tjenester som er drifta fra utlandet.

Datasystemer underlagt sikkerhetsloven må kjøre i Norge. Sett bort fra disse, har offentlige virksomheter de siste årene langt på vei kunnet velge om de vil skysette sine applikasjoner. Brorparten har imidlertid intern drift eller kjøper driftstjenester av en norsk driftsleverandør. Enkelte, som NAV og Lånekassen, har vært tidlig ute og kommet et godt stykke. Det må være disse som NSM er bekymret for.

Det finnes ikke et enkelt svar på om en nasjonal sky er en god idé, men her er noen betraktninger.

Tilbake til steinalderen?

Statens Datasentral har røtter tilbake til 1972, og EDB Fellesdata enda et tiår tidligere. Dette er selskaper som har gått gjennom privatisering, oppkjøp og sammenslåinger. Vi kjenner dem i dag som TietoEVRY, der EVRY er den norske delen av selskapet.

TietoEVRY tilbyr skytjenester, men blir ikke nevnt sammen med Azure, AWS og GCP. Tradisjonelle driftsselskap har aldri klart å ta steget opp i den øverste divisjonen. De tilbyr stadig bedre automatiserte og virtualiserte driftstjenester, men kan ikke konkurrere med de store på tjenestebredde, innovasjon, sikkerhet, skalerbarhet, selvbetjening, automatisering osv. Konkurransefortrinnet til de tradisjonelle driftsselskapene er at de har datasentre og tilstedeværelse i Norge.

Statens Datasentral 2.0

Det er åpenbart mulig å tenke seg en moderne nasjonal PaaS-platform. Offentlig PaaS Norge er et fagnettverk for offentlige etater og halv-offentlige selskaper som lager nye og moderne IT-plattformer basert på PaaS. Typisk er dette basert på Kubernetes-økosystemet. Det finnes et rikt og modent utvalg åpen-kildekode-produkter som kan gi plattformen en fin tjenestebredde.

En variant som diskuteres er å konsolidere de interne driftsmiljøene i offentlige virksomheter til en felles driftsorganisasjon. Teoretisk kan dette gi mening; hvorfor sitte på hver sin tue med folk og infrastruktur? Utfordringen her er nok bagasjen av etablerte systemer, være seg stormaskiner, databaser, skreddersydde kjernesystemer og liknende. Disse vil ikke kunne gaffeltruckes over på en moderne PaaS uten vesentlig modernisering. Slike legacysystemer er det mange av der ute.

Jeg har vanskelig for å se for meg at en nyetablert nasjonal driftsleverandør skal kunne levere vesentlig bedre driftstjenester sammenliknet med dagens norske driftsleverandører og intern drift. Hvis det var enkelt og mulig, hvorfor finnes det ikke allerede i dagens marked?

Den største utfordringen vil nok være styringen av et sånt samarbeid. Når var sist man hørte om et vellykket, mange-milliarders, toppstyrt samarbeidsprosjekt mellom mange titalls offentlige virksomheter? Hvordan og hvorfor kan dette lykkes bedre enn det man har klart hittil?

Etablering av en nasjonal sky vil nok også måtte skubbe seg på lover og regler for anskaffelse og konkurranse.

Hvis Norge velger å etablere en nasjonal sky, får vi krysse fingeren for at resultatet blir vesentlig bedre enn dagens interne driftsplattformer. De fleste som jobber med moderne softwareutvikling hadde satt stor pris på en automatisert, sikker, skalerbar og selvbetjent PaaS. Spørsmålet er bare hvor mye vi kan håpe på, ut fra historikken og forutsetningene for å lykkes.

Farvel, sky?

Vi som driver med moderne softwareutvikling har grått, lengtet og ønsket oss skytjenester i mange år. Mange offentlige virksomheter har utsatt skysetting lenge, og dette signalet fra NSM kan nok være kjærkomment for alle som ønsker fortsatt intern drift.

Et ikke helt utenkelig scenario:

Offentlige virksomheter stanser sine planer om skysetting i påvente av nasjonal sky.

Etableringen av nasjonal sky tar 2-4 år.

Nasjonal sky er klar for bruk. Får vi et A-lag av private virksomheter som benytter skytjenester fra AWS/GCP/Azure, og et B-lag av offentlige aktører på nasjonal sky?

Er det dette vi ønsker oss?

P.S. Denne artikkelen diskuterer ikke konsekvensene av SchremsII, som også vil påvirke hvordan offentlige og private virksomheter benytter skytjenester. Mer om det i en annen luke.

Statens Datasentral 2.0

Integrating security as a part of application development is desirable, but it's often forgotten or dismissed in practice. Dependabot is a Github feature that will help you keep all your dependencies invulnerable and up-to-date, and you can enable it in just a few clicks!

I've often been told that integrating security as a role and shared responsibility in application development is important. Yet, in my experience, security is frequently given a low priority by developers. Progress is primarily estimated by looking at an application's new features, and security measures are considered invasive and complicating to our day-to-day-work. Luckily, there are tools that requires little effort to increase our focus on security, such as Github's Dependabot

What is Dependabot?

Simply put, Dependabot is a tool designated to keep dependencies secure and up-to-date. How it works can be summarized in the following three steps:

Dependabot scans your dependency files for outdated or insecure dependencies.

Dependabot creates pull requests for dependencies that need an update.

You review, test and merge Dependabot’s changes.

Following Github's acquisition of Dependabot in May 2019, the feature was added natively to Github. With only a few clicks, Dependabot can easily be enabled from the Github dashboard, and help you keep your application’s dependencies up-to-date.

Getting started

Enough background information, let us talk about how to get started. Github's security settings can be configured for an entire organization, applying to all of the organization’s repositories, or it can be configured per repository. Which is the better option depends on your context. If you are the owner of your organization and it is desirable to enable the security features across all of its repositories, obviously it would be less time consuming to configure this on the organization level, rather than for each repository individually. However, if you are a part of a large organization with lots of teams and projects, enabling Dependabot on the repository level might be the only option.

In this example, we will enable Dependabot on a single repository. In your repository, there should be a button titled “Security”. Note that the security option is only visible to administrators, hence it will not be visible if you don't have the admin role in the repository.

In the security overview, there is a row titled “Dependabot alerts”. If not already activated, there should be a button to “Enable Dependabot alerts”. Click it. You should now have three options that you can enable:

Dependency graph

Dependabot alerts

Dependabot security updates

Dependency graph

The dependency graph is a summary of the manifest and lock files stored in a repository. Enabling this feature is a prerequisite for the other options, as Dependabot requires access to the dependency graph in order to create alerts and updates.

Dependabot alerts

The alerts allows Dependabot to notify you when it finds a weakness. This is probably the key feature that you are here for. When activated, the number of unresolved alerts is highlighted in the "Security" button that we previously clicked. On the security tab, the alerts highlighting currently outdated dependencies are listed. By clicking on an alert, you will find more details on it, such as a description of the vulnerability and in what version of the dependency it was patched.

Dependabot security updates

By enabling the third option, the security updates, whenever a vulnerable dependency is discovered, Dependabot will try to fix it. If it is possible to upgrade the vulnerable dependency without disrupting the dependency graph of the repository, Dependabot will generate a pull request bringing the dependency up-to-date. The pull request is given a compatibility score indicating whether the update could cause breaking changes. The number is based on the percentage of tests in public repositories that passed when performing the same update.

Unfortunately, in my experience working with Dependabot, more often than not, it is unable to generate pull requests. This is often because the source of the vulnerability is an indirect or transitive dependency. The Dependabot documentation states that “(...) security updates are triggered only for dependencies that are specified in a manifest or lock file. Dependabot is unable to update an indirect or transitive dependency that is not explicitly defined.”. This means that even though Dependabot is able to perform some automatic updates, you should expect to perform most of the updates yourself - for now.

Even though Dependabot is unable to perform automated dependency updates most of the time, I would recommend enabling the feature. Occasionally it will be able to create the pull request for you, and you'll have saved yourself a couple of minutes that you can spend on something else.

If you already have established an effective process to update your project's dependencies, Dependabot might not be able to help you. Otherwise, you should definitely give it a try. It's free, it takes minimal effort to set up and - no matter how you use it - it's a better option than not monitoring your dependencies in any way.

Github Security: Getting started with Dependabot

This is an opinionated post. Consider yourself warned. You will probably disagree with at least some of it, and that is completely fine. The important part is the thought process – don’t just read through it and agree or disagree, think about what you agree or disagree with and why. Becoming aware of these things is valuable: once you become aware of why you dislike something, you can start doing something about it.

Over the years, I have noticed there is a small set of features I value more than others in programming languages. These features allow me to express myself in a concise and clear manner, makes my code more readable, and minimizes the amounts of errors I make. They are not "magical" in any way. Far from it. Rather, they are more.. fundamental aspects of the progamming language itself. Let’s dive in!

Expressive type system

Types help me reason. Explicitly typed code communicates intent better, and is easier to navigate without losing context completely. Type systems comes in a variety of flavors, ranging from the most simple ones to more advanced systems with gradual typing, dependent types, type classes, and so on. Although I personally prefer more advanced ones, I believe this is more of a subjective topic. And of less importance, really. The main benefit of type systems exists in even the simplest ones: increased safety and readability. Remember: the compiler is your friend – the more it can verify, the better!

Type inference

I was a bit unsure if this point had its place in this blog post. On one hand, I feel its more of a convenience feature than strictly neccessary. On the other, an overly chatty and boilerplaty type system can cause more harm than good – if your business logic drowns in types, it’s too far in the wrong direction. Java, around the 1.5 era, is an example of this. In addition, type inference itself is a a double-egded sword. If the compiler inferes a different type than what you would have thought, it can be quite hard to spot where the actual error is. This is the reason some languages and communities define best practices around type inference, like being explicit in function signatures. Anyway. When I find that perfect spot between explicit types and inferred types, it just makes me really happy.

Algebraic data types

I remember the first time I enountered sum types. My first response was «but.. Isn’t this just some kind of enums?». I can kind of see why I thought that, but I feel this is a limiting way of approaching them. Where enums helps me limit valid values, algebraic data types is so much more. It enables me to model my domain and state in a precise and correct manner, and makes illegal states «impossible». This didn’t really become clear to me until I went back to Java. I find modelling without them cumbersome and hard, and sometimes even end up in situations where I can’t avoid impossible states. Few things are more confusing than finding the comment «this can’t happen» in a branch of someones code.

First class functions

I don’t even understand why this is subject to discussion anymore, but functions should be treated just like any other value [^1] in a programming language. Returning functions and accepting functions as parameters is a fundamental feature to create abstractions and solve problems with composition. This just isn’t up for debate – I would take first class functions over any other feature any day of the week!

[^1] – I use the term "value" loosely here. I don’t mean to debate value types, reference types, and pass by value or reference. Those are implementation details. The point is to see what advantages you get when a language treats simple values and functions the same.

Functional Programming

Functional programming taught me to value these features. Sure, you will find imperative languages with these features as well, but they never feel like a core feature of the language. Even Java comes with some sort of primitive type inference these days, but it mostly feels tacked onto the language.

With some notable exceptions (looking at you, LISP), these are core features in all functional programming languages I use. They make my day better, so I would think hard before adopting a language without them these days.

Which feature do you miss or disagree with the most?

2-Dec

Today's posts