Some things in life are certain - for instance, we tend to take for granted that the sun will rise every day. Similarly, everyone knows that each Christmas, Canadian-Italian singer-songwriter Michael Bublé will emerge from his secret hideaway and top Christmas playlists yet again.

However, other things are not so certain. Today, we will take a look at how uncertainty may play a role in how we interpret machine learning models.

###Black Box versus White Box Models

As modern machine learning models become able to solve increasingly challenging problems, their application has grown tremendously. In the US, deep learning patent applications grew on average by 175% annually between 2013 and 2016. This has caused an ever-expanding range of complex models.

One challenge with the complexity of modern deep learning models is the non-trivial task of understanding precisely why a model produces the outputs it does. Deep learning models may have millions of model parameters and determining the values is not an exact science. Once trained, understanding the complex relationship between inputs and outputs can be extremely difficult. In other words, many modern models exhibit traits of a black box. In black box models the relationship between inputs and outputs is not easily interpretable – it’s as if we are dealing with a black box which magically gives us answers given our inputs.

Conversely, in white box models we can clearly understand and explain the relationship between inputs and outputs, and thus also model behaviour.

So why can black box traits be problematic? There are several reasons:

Transparency: Certain applications require a business to be transparent about its conduct. For instance, Norwegian insurance companies are required by law to be able to document why a customer was given a higher than average premium on their car insurance. This is not possible if the premium is the output of a black box deep learning neural network.

Interpretability: As data scientists, we are often more concerned with how a deep learning model performs than we are with the exact science of the underlying algorithm. However, if you know your deep learning model in a critical application you are able to also improve on other parts and be able to critically evaluate why the model performs as it does.

Explainability: As practitioners of machine learning, we try to solve real world problems. In many cases, deep learning models support human decision making or are applied in settings where the end user may not be a data scientist. Usually, we see that the level of trust in a machine learning application increases with our ability to explain a model’s behaviour in human terms. And if your model isn’t trusted by its users, it may never actually be put in production.

The debate of which is superior between white and black box models is not something we will dive into in this blog post – but we will look at how we can utilize uncertainty to assist in both interpretability and to some extent explainability.

###About uncertainty

Uncertainty in the setting of modelling a problem comes in two forms:

Aleatoric uncertainty refers to irreducible uncertainty in the variables or data we are trying to model. Consider a model which predicts the outcome of throwing a dice – no matter how many data points we gather, there is inherent uncertainty in the outcome which we cannot reduce. In fact, aleatoric stems from Latin aleator, meaning dice player.

Epistemic uncertainty is often called model uncertainty and refers to the uncertainty that we have chosen the correct model for our problem. This uncertainty is reducible given more data. Using our dice throw prediction model again, let’s say we are given data points for throws of 2, 3 and 5. At this point, a careful data scientist knows that there is a possibility the data was generated by an 8-sided dice. However, if we are given 1 million data points and all fall between 1 and 6, we are confident the 6-sided dice is the right model – the epistemic uncertainty is reduced to virtually 0.

Thus, for every prediction a model makes we deal with both aleatoric and epistemic uncertainty, in sum comprising what is often referred to as prediction uncertainty. As a side note, if we consider humans to be deep learning models in action, we also deal with the same uncertainties. Below is the picture of “The Dress”, a picture which went viral because people couldn’t agree on what colours they see (spoiler: it really is black and blue). We’ll leave it to the reader to determine if this is aleatoric or epistemic uncertainty – or both!

###Uncertainty in a Machine Learning Problem

Let’s look at the two types of uncertainty in the context of a simple regression problem. Here we have some data points in two dimensions, and we wish to model the output y as a function of x. We have fitted a deep learning model to the data (a simple model would have sufficed for this problem, but the discussion scales to higher dimensions and more complex problems).

Our model seems to fit the data reasonably well, but there are two areas of particular interest.

In area A, it seems we have several values of y for similar x. Perhaps our data stems from an inconsistent sensor, or we are even missing a third-dimension, z, which could explain this behaviour. However, if we assume that we cannot repair our sensor or measure more dimensions – we are dealing with high aleatoric uncertainty in this area. Ideally, we would want the model to tell us that on average its prediction will be fairly accurate, but real values of y may deviate more in area A than in its immediate neighbouring spaces.

In area B, we lack data altogether. Yet, if we ask our model what it predicts for y in this area, it will give an answer. Now you might object that in the same way you wouldn’t ask your dentist for advice on stock trading, you shouldn’t ask this model what y is for unseen x. However, in real applications we may not always be able to guarantee this doesn’t happen – consider for instance an autonomous car image recognition system seeing an object for the first time. In area B, we are very uncertain that we have the right model and the epistemic uncertainty is high. However, given some more data in this area, we could reduce it. Here, we would want the model to say (like the dentist would): “I’ll give you an answer if you force me to, but honestly I have no idea”.

Clearly, while uncertainty will not improve the performance of our model by itself, it will help with explaining and predicting the behaviour of the model. In areas with high prediction uncertainty, we expect our model to be wrong more often – traits which by themselves are not explained by traditional performance metrics. Effectively communicating this can improve trust in a system even though performance remains constant.

Now that we have identified these uncertain areas, let’s briefly look at how we can estimate the uncertainty and communicate it using deep learning models.

###Estimating Uncertainty

Data scientists with a penchant for the Bayesian school will argue that you should always model uncertainty because it is always present in any problem. However, we can often afford to overlook it if the application is not critical. In cases that you do need to consider uncertainty, several approaches exist. A thorough review is beyond the scope of this blog post, but we will give some pointers for those who are interested.

For epistemic uncertainty, we will highlight two approaches:

Ensemble methods: This approach works by training an ensemble of models, with randomness either in parameters or the learning process leading to slightly different fits. Treating these models as a distribution then lets us estimate the epistemic uncertainty. As a generic approach, this works for any model for which there is a randomness to the training process.

Dropout: Dropout is a technique typically applied during the training of deep neural networks, where neurons are randomly dropped each iteration to increase robustness to overfitting the data. However, it was recently shown that dropout can be used at evaluation time to approximate model uncertainty. Thus, instead of an ensemble of models we use small variations of the same model to estimate epistemic uncertainty. See the excellent blog post by Yarin Gal below for the derivation and interactive examples:

For the aleatoric uncertainty:

Learning the aleatoric uncertainty: Under some assumptions on how noise in the data behaves as a function of the input variables, we can modify the loss function of our model to include an estimate of the aleatoric uncertainty. For image recognition where a model learns the depth, i.e., the distance to objects, we can output higher aleatoric uncertainty for objects with blurred boundaries or objects that are seemingly very far away. For regression, we can output our prediction as a distribution with a mean and variance.

Also, worth mentioning is the study of using machine learning to produce prediction intervals for regression problems which is concerned with outputting intervals with a given confidence level that the real value will be contained within the interval. Some approaches do not require assumptions on the underlying distributions of the data.

###Does This Mean a Free Lunch?

Finally, we should point out that in data science, there is no free lunch. This also applies to estimating uncertainty in deep learning models, which comes at a cost:

The complexity of a model often increases if we for instance alter loss functions to take into consideration noise in the data. Thus, it may take more time to train models

Ensemble methods and the dropout approach require several iterations at test time, which means it can take longer time to run models and obtain predictions

Some approaches require assumptions regarding the nature of the aleatoric uncertainty, which may be difficult to justify in complex settings where we do not understand the data well enough

We encourage you to remember that uncertainty is always present in deep learning models, so determining when you need to take it into consideration is a valuable skill as a data scientist.

Uncertainty in Machine Learning

Har du noen gang gjort noe bare fordi andre gjorde det? Trent mer og spist sunnere i januar? Eller kanskje til og med blitt værende i et forhold du visste burde vært slutt? Da er du ikke alene. Men hvorfor gjør vi egentlig disse tingene? Og hvordan kan vi bruke kunnskap om adferd i designprosessen? Roxy og Gerome i Coglode lager Nuggets for å hjelpe oss med dette.

Hva er Nuggets, og hva kan man bruke de til?

Nuggets er en “liten munnfull” adferdsforskning formulert som et prinsipp. Coglode ønsker å gjøre forskningen enkel for folk flest å forstå og benytte seg av. Derfor oppsummerer de den på praktiske kort som de kaller Nuggets.

Hver Nugget består av et navngitt adferdsprinsipp, med en beskrivelse av hva det går ut på. Kortet inneholder også en oppsummering av forskningen bak prinsippet, og hovedfunn som forklarer hvordan man anvender det. Navngivningen er viktig for at de skal være enkelt å snakke om, og henvise til. "Sosialt Bevis" (Social Proof) er et eksempel på en slik Nugget. Det handler om at vi kopierer andres oppførsel, spesielt i ukjente situasjoner. Dette prinsippet har blant annet TripAdvisor benyttet seg av, og lyktes godt med.

Nuggetsene er laget for å gjøre det enklere å ta beslutninger, bli inspirert, og få frem gode idéer raskt. Det som skiller dette fra andre kreative verktøy er at det bygger på vitenskap, og kan tilføre ny kunnskap som hjelper teamet å ta gode valg. For å ta de i bruk i designprosessen begynner du med problemstillingen som skal løses. Deretter velger du ut noen Nuggets du kan bruke for å komme på nye idéer og konsepter. Det er viktig å teste idéene du lager for å se om du får den effekten du ønsker. Den beste måten å teste om en løsning fungerer er å kjøre en eller flere A/B tester.

Hvordan nudge flere til å gi tips til Foodora-syklistene?

I høst kjørte faggruppene “Anvendt Psykologi og Nevrovitenskap” og “Kreativ Problemløsning” i Bekk en felles workshop der målet var å bli kjent med startpakken (Box One) vi har kjøpt. På den tiden var det mye snakk om Foodora, og lønningene til sykkelbudene deres. Derfor tenkte vi at det kunne være gøy å kjøre en workshop hvor vi var på syklistenes side, og lekte oss litt med problemstillingen:

“Hvordan få flere folk til å gi tips/mer tips til Foodora-syklistene når de bestiller mat fra Foodora?”

Agendaen for workshopen var ganske enkel:

Vi delte oss opp i grupper. For å kickstarte workshopen kjørte vi oppvarmingsøvelsen “Zombie cats" som Therese skrev om 8. desember.

Etter oppvarmingen fikk alle på gruppa velge seg en Nugget å bli kjent med, som de så forklarte til resten av gruppa.

Vi fortsatte workshopen med individuell brainstorm på Post Its, der vi brukte Nuggetsene som trigger og inspirasjon.

Etterpå presenterte vi idéene til hverandre, og bearbeidet de videre i gruppen. Hver gruppe måtte velge to idéer de hadde tro på, tegne de på A3 ark, og henge de opp på veggen.

For å simulere en test av løsningene, kjørte vi avstemning. Alle fikk tre stemmer hver, som skulle brukes på den løsningen vi tenkte ville ha mest innflytelse på oss. Valget var selvsagt hemmelig, slik at vi ikke skulle påvirke hverandre.

Vi kom på mange forskjellige, og kanskje litt uventede idéer til hvordan vi kan få brukerne av Foodora til å gi sykkelbudene mer tips. En av idéene var inspirert av Nuggeten “Gjensidighet” (Reciprocity): Når noen er snille med oss, ønsker vi å være snill tilbake. For å øke sannsynligheten for at sykkelbudet skulle få tips, tenkte vi å få kundene til å føle at budet var snille med de. En måte å trigge det på var å legge frem hvor kaldt og guffent vær syklisten måtte ut i for å levere maten.

En annen idé kombinerte Nuggetsene “Overraskelse” (Surprise) og “Belønning” (Reward). Vi mennesker responderer bra på positive overraskelser, og endrer oppførsel når vi får gaver som forsterker handlinger vi gjør. Med de prinsippene som inspirasjon kom vi på idéen om å gi de brukerne som gav tips en overraskelse. Overraskelsen besto av en hyggelig melding, og beskjed om at de ville få en gratis dessert som takk ved neste bestilling.

Konseptet som vant avstemningen bygget på Nuggeten “Status”: Vi ser stadig etter måter å forbedre hvordan andre ser oss. Konseptet gikk ut på at kundene fikk maten levert i ulike poser avhengig av hvor mye tips de hadde gitt. Hadde de droppet tips fikk de rød pose, og hadde de gitt bra tips så fikk de grønn. Etter hvert som folk ble kjent med dette, ville de som hadde grønn pose bli sett i et bedre lys enn de som hadde rød.

Så.. hvordan synes vi det var å jobbe med Nuggets?

En del av løsningene vi kom opp med kan vel sies å være noe kontroversielle. Men målet med workshopen var å teste ut hvordan vi kunne bruke Nuggets i workshop, og ikke løsningene i seg selv. Så derfor spurte jeg noen av deltakerne hvordan de synes det var:

“Det var fint å bruke Nuggetsene i idégenereringa for å tvinge hjernen til å tenke annleis, og få andre koblingar. Blei skikkeleg gira av å bruke verktøyet i gruppa, me kom fram til så mange bra idear!” – Åshild Aaen Torpe, Interaksjonsdesigner



“Jeg synes adferdsnuggets var veldig interessant fordi jeg kom ut av mine vante tankemønstre, og kom på helt nye og spennende ideer. 10/10. Would recommend.” – Alexander Slåfjell, Utvikler



“Jeg syntes det var superinteressant 😀 Et snedig verktøy for å komme opp med ideer på kort tid – noe folk fant i fleng 👍 – Erik Mathisen, Tekstforfatter

For å oppsummere;

Nuggets er ypperlig for å komme frem til kreative løsninger. De kan hjelpe deg når du skal gjøre endringer i en tekst, så vel som når du skal utforme idéer og konsepter. De er like nyttig når du er alene som sammen med andre. Ønsker du å lære mer om og teste ut Nuggets, ligger mange av de beskrevet på Coglode sine nettsider, eller du kan bestille startpakken her.

__

Illustrasjonene og bildene av Nuggets er hentet fraCoglode.com

Bygg bedre produkter ved hjelp av Nuggets!

In a phishing attack the attacker will try to steal user data, e.g. login credentials. Reverse tabnabbing is a phishing method, and here we will try to explain what it is and how it can be prevented.

Websites often refer to pages on other websites. Such links are some times opened in a new tab. If we add the target="_blank" to an a-element in HTML, the link will open in a new tab. We can also achieve the same by using Javascript. The page we link to might be either safe or unsafe. We cannot know since we have no control of it.

A linked page opened with target="_blank" or by window.open() in Javascript, the linked page will have access to the same window.opener-property as the linking page. Thus, the linked page can set the property window.opener.location to anything it wants. That opens a set of possibilities, and we can imagine the following attack scenario:

The attacker creates a page with whatever content and share it with someone on a social network, which opens the link to the attacker's page with target="_blank".

In addition, the attacker creates a login page looking identical to the social network login page.

On the page shared on the social network, the attacker puts window.opener.location = <url to fake login page>.

The fake login page appears, and ask the user to re-enter credentials. Being prompted for login credentials happens from time to time, so the user does not think too much about it.

Voilà. The attacker has got the user's login credentials.

Solutions

There are two quite easy fixes to prevent this kind of attack.

Add rel="noopener noreferrer"to every a-element that has target set to "_blank". noopener ensures that the linked page does not have access to window.opener from the linking page. noreferrer make sure that the request referrer header is not being sent. Thus, the destination site will not see the URL the user came from. According to caniuse.com, the support for noreferrer and noopener is good in recent versions of major browsers. Be aware that Internet Explorer is the usual exception.

If you are using Javascript you can achieve the same by setting the opener-property to null.\ var myNewWindow = window.open(url, name, 'noopener,noreferrer')\ myNewWindow.opener = null

If you are showing user-generated content on your page you must sanitize the input and apply "noopener noreferrer" to every link.

We hope that you have this in mind when you develop your websites. Please refer to the links below if you want to know more.

Reverse tabnabbing

Have you ever had the need for scheduling background tasks but always wanted a more lightweight alternative to Quartz Scheduler?

Introducing a lightweight alternative

Running background tasks is an important part of backend software. In the Java ecosystem, one of the most used schedulers for running background tasks is the Quartz Scheduler. It is simply a beast in terms of functionality and can be in form of complexity.

Gustav Karlsson (Bekk) longed for a more lightweight alternative himself, one which did not require 11 database tables to function. He wound up writing his own alternative scheduler – db-scheduler – which just requires a single database table but stills support clustering and embedding. Even though the API is simple, it still provides some pretty powerful features.

Give me an example!

The library itself does not have any bindings to other libraries or frameworks but a starter is provided for Spring Boot projects. Let’s use that!

Assuming a blank Spring Boot projects, you’ll need a few dependencies in your pom.xml (for Maven):

After that add the following in src/main/resources/schema.sql. For embedded databases, it will automatically get executed.

Go on to tweak your application.properties:

Finally add a new Spring Configuration class that will provide the tasks that need to be executed. These are exposed as regular beans, so you can wire in whatever you need.

Running the app should now produce log lines like:

Summary

As the example shows, it is really easy to get some basic background jobs up and running. db-scheduler also supports more advanced features such as scheduling one-time tasks with arbitrary data for use at execution-time or recurring tasks keeping an internal state.

Persistent job scheduling without Quartz

Today's interview is with @mikaelbrevik. Mikael is an Open source maintainer, the organizer of @_bartjs , host of the podcast @kortslutningpod, video-blog @kodesnutt and works at @variant_as.

The questions below came from a simple question we asked ourselves:

if you had the chance, what would you ask an Open source maintainer/person/community member.

Here we go!

What are your biggest challenges in working with Open source today and how do you handle them?

There are different challenges with different parts of Open Source. One challenge is Open Source sustainability in hectic everyday situations. We are often under constant pressure to produce features and progress, and especially as a consultant, it can be difficult to take time to contribute to projects. This also ties in challenges with funding Open Source projects. We can't rely on people doing free work and taking advantage of this. Another challenge is respecting licensing. We are way too focused on the ability to use something and not enough on legality. Just because I can install something from a package manager, doesn't mean I'm allowed to.

We need to be more aware of different licenses, compatibility, and limitations. If I had 10 NOK every time I heard "it was open on the internet" or even "I don't know" when asking "what is the license", I could have funded several Open Source projects. One way to raise awareness for this in your project is to add license whitelisting as a CI step.

On a personal level, the biggest challenge I have is having energy and time to maintain projects. It doesn't matter if you have a lot of time when starting a project if you don't have the time when it gains traction. It can take a short, intense, time to create a project, but several years to maintain it. As it often is, life occurrences come in bulk. I can promise you that if you have several Open Source projects, there will be times when all of them get issues at the same time. At some point, you get so many issues that you get behind on them. From there, it is a bad spiral of stress and guilt.

Coming to terms with the fact that some people will be frustrated, and some times angry with you, can be tough. There are different ways to handle this, both technical, and personnel changes. Getting used to the feeling that sometimes you just have to let it be, even if other people have issues, is the biggest one for me. Also, closing stale issues can have a huge impact. The absolute biggest impact is to get other people involved to share the workload. This cannot be underestimated. Both for the quality of the project, but also for your wellbeing. I have had good success with just adding people showing interest in the project as collaborators, and give them the access and input they need to feel included.

Are there any useful tools you use to make your GitHub-life easier to manage, and if so, could you share a few of them with us?

I find myself using fewer and fewer tools. I'm guessing there are several things I could do to improve my Open Source life and especially maintaining, such as bots (auto-close stale, code quality checks, etc), but there is also some work tuning them to work in your favor and not create more work than it relieves. The most important part for me is notifications and email-client. Having the "inbox zero" approach and snoozing emails as tasks is mostly how I structure Github Issue work. It is also probably the biggest cause of guilt.

How can repository-owners facilitate contributions from newcomers in Open source?

As I mentioned earlier, I think it all starts with including people. Adding them as collaborators and making them feel their input and work is valuable. Having automation steps with tests and feedback to pull requests is always a quick win, even though it can take some time to set it up. Make sure tests actually pass and that there are tests, to begin with.

Put some work into minimizing the development setup. And of course have up to date documentation for how to contribute, architectural choices and how the project works. Having automation for formatting (e.g. prettier and other code formatters) goes a long way to guide new contributors for code style without having to seem nitpicky and pedantic. Guide people through pull requests instead of acting as a gatekeeper. This doesn't mean you should accept every PR, but it says something about how you should communicate. The most important thing of all, of course, is: Be kind.

If you could time-travel to when you first started, what advice would you give yourself as an Open source rookie?

Oh, where to start. And where to stop. A lot of this ties in with what I talked about in previous questions also. I think developers, myself included, often look at Open Source as a way to be prominent. It's kind of a popularity contest, where we idolize people of prominent projects, and projects from prominent people. I think one thing I would say is: Not all contribution needs to be grand or at "high impact projects". There are a lot of different levels on how to contribute, and adding big features isn't the most important one. Don't be possessive of the projects you create. Stars, downloads, and likes to feel good, initially, but having a healthy project over time which has value feels better.

This isn't advice solely for me in the past, but just as much for the present and future me. But of course, this doesn't mean you should experiment with Open Source and release whatever makes you happy. And Open Source can be amazing. Both for achieving things and for learning. But include other people early, rather than later.

Can you describe how you see the future of Open source, what new directions or paths do you think it will take?

The future of Open Source is like Will Smith and his trusted dog walking alone down the main street. You have all the time in the world, though somewhat lonely. But midway you find that the world exists of hungry zombie vampires chasing you, and the story takes a whole different direction. Mostly just a funny analogy, and I don't really think Open Source will be dystopian. But I think some challenges will be more and more apparent. Mostly funding.

In the front end world, we see more and more ads, sponsored content, and donation pleas with Open Source. And there is a reason for this: Open Source should not and cannot be free work. We are seeing the more formalized way to handle this (Github Sponsorship, Open Collective, Tidelift, etc), and I think this will continue to create different models for doing Open Source. I think Open Source is more a way of thinking and a methodology rather than a technical thing. And as such, I think Open Source will continue to spread to different parts of our field. Not only for code but for practices and information. E.g. at Variant we have Open Source customer and employee contracts, which honestly I think is a huge leap masked as a small step. The thought that actual business contracts used regularly are available as MIT license, open for reuse by others and open for change, I think it is very interesting.

In my mind, this is representative of what Open Source is and can be. A fundamental protocol and precedence for collectively, across companies, to improve as an industry. I've also often spoken about a lack of design (in all shapes and forms) in Open Source. There are many products, libraries and third party projects we use that should have different perspectives. Both for universal design and general design. This also goes for diversity in contributors. And I think as with the rest of the industry, this is and will be more and more important.

Thanks for taking the time to talk with us Mikael! 💪

Interview with Mikael Brevik

Tail recursion is a special way of writing recursive functions such that a compiler can optimize the recursion away and implement the algorithm as a loop instead. This is not because loops are inherently faster, but because every function call generally incurs the cost of a new stack frame in which it executes, sometimes leading to the dreaded stack overflow - where the beloved site of the same name, gets its name. Let's investigate how this works.

But first we need to consider what recursion is and which functions are tail recursive and which are not. A function is said to be recursive if it is defined in terms of itself. Not only is recursion a natural and elegant solution for many problems, but in pure functional languages such as Elm and Haskell, there is simply no other way to express looping. As is customary when thinking about recursion, we will look at the factorial function. In mathematics we might see a definition such as n! = n * (n-1) * ... * 2 * 1. If the author cares to define the same function recursively you might see

Notice how the definition is made of two parts, a recursive part and a base-case part. This translates nicely to programming. In a functional language we might define the factorial function like the following.

The syntax is slightly different, but the structure is suprisingly similar. This function is however not tail recursive, it is merely recursive.

A slightly re-written version, which is tail recursive, might look like the follwing.

This definition is mostly the same, with the addition of a new parameter acc, short for accumulator. Introducing an accumulator is often necessary to be able to rewrite a function to be tail recursive. Annoyingly, this version has to be called with an initial value of acc=1, so for instance 5! would look like factorial 1 5.

The crucial difference between these definitions, however, is that the recursive call to factorial in this new definition is in "tail position". In a sense, the last operation that is performed in the body of the function definition, is to call itself. This is what allows the compiler to replace the function call with a loop instruction and reuse the same stack frame - because there is no more computation to be done in this stack frame after the recursive call. When it returns, all we do is return that answer. In which case we might as well return the answer directly.

Maybe this becomes more clear if we take a look at the same two definitions re-written more verbosely, using intermediary variable names and if statements, instead of the more terse mathematics-like syntax above.

The important difference is that in the first definition there is more work to be done after the recursive call, namely the multiplication n * rest, while in the second definition, all the work happens before.

One way I like to think about this, is in a mutable way. I think of the last line factorial sofar next as an instruction to go back to the top of the function and bind acc to this new value sofar and n to next. Whenever n == 0 and it is time to break out of this looping recursion, we simply return the latest value of acc directly to the original caller of the factorial function.

In fact, this is precisely how Clojure implements tail recursion. Instead of detecting it automatically, Clojure forces you to use a special syntax with the special keywords loop and recur. Here is a Clojure implementation.

Again we see the same structure, only expressed using a different syntax.

Trampolining

In many popular languages such as Python and JavaScript there is no implementation of the language which implements tail recursion, so even if you write your functions in a tail recursive style, it has no effect.

We can however implement it ourselves if we want to do many and deep recursions, without blowing the stack, using a technique known as trampolining. In this technique we write our functions in a tail recursive style, but instead of evaluating our function recursively before returning, we instead return the function we want called and the argument we want to call it with. Then we wrap this in a loop which is responsible for calling it repeatedly, until the function signals that it is done. We signal this by returning a special value, such as undefined, instead of the function, when we reach the base case.

Here is an implementation in JS.

fact can now be called with [1, 5] to evalueate the factorial of 5. One thing to note about this example is the use of two-element lists to represent tuples. The return value of factorial is a tuple of the function reference (or undefined) and a tuple of the parameters to the function. The parameters of the function is the accumulator and the current value to compute the factorial of.

Notice also the destructuring of f and x in the while loop, which also re-assigns the variables of the same names. For every iteration of the loop, x is the next set of parameters the function will be called with. When the loop is finished and the recursion done, the value of x is the final value of the computation, the logical return value of the recursive function.

Tail recursion

Once upon a time, a developer named Dag was asked to create a Christmas card for his team. Having little experience with photo editing software, he turned to his main strength: web development.

He started out collecting some photos of his team. He considered many options. Should he find them on Facebook, take some himself, or find some funny characters to illustrate them? While pondering about this, he opened a pull request to give some feedback. Looking back at him from the pull request was the answer!

"Why not use the profile pictures on GitHub? They are already uploaded and can easily be scaled!" So he did just that, collected the GitHub profile pictures of everyone on his team.

Sphering it up

"That is a good start," he thought to himself. But where do I go from here? He remembered Mira on his team styling the user profile images in their project, so he went and asked her how she had done it.

"How I styled those rounded images?" Mira said. "That is quite easy - you can use border-radius 50% to get a circular image!"

Dag added the following styling to the images he had collected.

"This is looking nicer already," he thought to himself.

Being the curious soul that he is, Dag had to look into what the CSS did. Looking up his trusted MDN web docs he found MDN web docs: border-radius

"Neat!", he thought. "This can be very useful for anything that needs rounded corners. Even if the items have variable sizes as I can use both lengths in px and percentages. I can also create elliptical shapes and have control of each corner. But for now, I'll stick with the one value to rule them all".

Cutting corners for Christmas

"But this is still not a Christmas card," he thought. "We need to make this more christmassy. " Had he not just read something about clip-path? He had thought that "that sure looks nice, but when will I ever use it?". He figured he could give it a go. He looked up the documentation; he even checked it twice. He was going to make sure he could figure it out. But even after looking at the documentation multiple times, it was hard to do what he wanted to do.

In the end, Dag looked up a clip-path generator Clippy — CSS clip-path maker as he was not satisfied with anything less than the best Christmas tree (shape)! After generating the path and cleaning it up. He could take a step back and look at the tree he had created.

And to think that all it took was a little code.

Dag tried to summarize the property, but quickly found he would have to reference the documentation when needed. But he noted that one could use different geometric shapes as well as SVG to create a clipping region of what parts of an element to show.

MDN web docs: clip-path

Decorating the tree

After taking some time to enjoy his hard work, Dag thought to himself, "Let's put it together." He added the images of his team to the tree and using positioning, positioned them on the tree as a Christmas decoration. "Look at how nice I've hung them on the tree," he thought to himself, followed by "... Uhm, that was a poor choice of words." He also added a fitting background to make it all come together even better.

"This is starting to look like something! But we should have a star too!". Using the same steps as with the tree, he created a star and placed it on the top of the tree.

"Better! But not quite there." The tree looks a bit bland. "Maybe a gradient could spice up the tree? But I've never gotten the hang of them". He took to his favorite search engine with a quack and quickly found something he thought he could use. CSS Stripes Generator. Generating some rad stripes and then tweaking them, he came up with a result he was happy with.

The background-image property turned out to be powerful, as one could get an image generated based on gradients.

All the CSS needed to generate some nice stripes was:

He noted the following as his own reference for how to use the linear-gradiant:

He put his hands behind his head, leaned backward, and enjoyed his work. But he only got to enjoy it for a short while as he noticed what time it was. "I'll finish this tomorrow," he thought as he made sure his hard work was saved.

To be continued tomorrow.

CSS Christmas card - Part 1: Cutting corners 

Have you ever wanted to host a low traffic website in Azure, but required features like SSL and a custom domain? Were you disappointed when you found out that these features are only supported in the B1 App Service Plan for a whooping 50$ a month? With some simple changes and a Content Delivery Network (CDN) you can have it all for FREE!

The easiest way to host a website in Azure is to use an Azure Web App backed by an App Service Plan. However, the free F1 App Service Plan is pretty basic. It’s shared infrastructure giving you 1 GB memory and 60 minutes a day of compute time. It has no support for custom domains, but provides SSL on the default *.azurewebsites.net address.

What if you want to add a custom domain to your free Azure website and still want to have SSL support? Can it be provided for free in Azure without using third party services? The simplest solution is to upgrade the App Service Plan, but that won’t be free.

What we want to achieve:

Free website hosting

SSL support

Custom domain support

Use only Azure services

Static website + Content Delivery Network

A common way to solve this problem for web applications where you have a separate backend and frontend, is to host the frontend as a static website and combine it with a Content Delivery Network (CDN) that can provide SSL on a custom domain. Netlify, GitHub Pages and Amazon Web Services (AWS) all have solutions to this problem and can provide this functionality for free. AWS has had this capability for years by combining the Amazon CloudFront CDN with Amazon S3 storage which can host a static website.

Basically it works like this:

The CDN manages the certificate that is used to encrypt the connection to the end-user with a custom domain and the connection between the CDN and the website is encrypted with the SSL certificate supplied by the website (if any).

Turns out this is now also possible in Azure. In late 2018 Azure announced support for hosting static websites in storage accounts. You can enable “Static website” on a storage account and get a special $web container where you can put static website content (HTML/CSS/Javascript and images) and have them served to the user as if it was a web server. The cool thing about this is that storage accounts supports custom domains for free.

When you enable the static website feature on a storage account the website will get an endpoint like:

https://<storageaccountname>.z13.web.core.windows.net

Azure CDN

There are many CDN providers that can provide free SSL on a custom domain, but with a pure Azure solution only the Azure CDN will do. The main benefit of using an Azure service is that the provisioning and configuration can easily be scripted using Azure Powershell or ARM-templates.

To try it out, you first have to create an Azure CDN profile in the Azure Portal and choose a pricing tier which supports custom domains. I chose the “Premium Verizon” tier just because it is the only tier that supports redirecting http => https using custom page rules. The CDN cost for a static website, which is typically just a few MB, is basically zero. Continue by adding a CDN endpoint inside the CDN profile that maps your CDN endpoint to your site:

<sitename>.azureedge.net → https://<storageaccountname>.z13.web.core.windows.net

As long as http is allowed on the storage account you should after a short while be able to access the site using the *.azureedge.net address. To add a custom domain to the CDN you first need to login to your domain host and add a CNAME mapping to your CDN endpoint like this:

www.mysite.com → <sitename>.azureedge.net

After the CNAME has been added and the DNS has propagated which in my case was pretty quick, Azure will allow you to add the custom domain to your CDN endpoint. When the custom domain is added, the last step is to enable SSL on the custom hostname which is just a flip of a switch. Just click the hostname and choose to enable custom domain HTTPS.

Azure will then generate a certificate for your domain that is valid for a year and will be automatically renewed. Enabling SSL can take quite a bit of time so expect to wait a day or two.

End-users will then be able to use the custom domain to access your website through the Azure CDN. But what about the backend? The storage account will only serve static assets so you still can’t have a custom domain for your backend. As long as you only need a SPA the backend api can be either a regular Azure website using the free tier or an Azure Function using consumption plan (which is partially free). The backend api will use the regular *.azurewebsites.net address which supports SSL and does not need to be served from the CDN. The end-users will not see the backend url since the api is only used by the javascript in the frontend. You only have to make sure to update the CORS settings for your backend api to accept requests from the custom domain.

Wrap-up

Using a combination of Azure CDN and an Azure Storage Account as your frontend and an Azure WebSite (or Azure Function) as your backend you can have a virtually free website on Azure that supports SSL encryption and a custom domain. You only need to pay for the storage and network traffic to the storage account, but that likely amounts to pennies a month. The whole infrastructure setup can also relatively easily be automated using ARM-templates.

The only major drawbacks are that the Azure Website free tier backend is limited to 60 CPU minutes a day and that the web app resources on the free tier are shared with other Azure customers. The site won’t support a lot of simultaneous visitors, but for a low traffic website that probably won’t be a problem. You can always pay to scale up the App Service Plan when needed.

Host a website in Azure with a custom domain + SSL for free

Conceptually, coroutines works about the same way as threads do, as both are asynchronous creatures. Just like threads, if you start a coroutine from the main thread, the main thread will keep going while the coroutine will happen asynchronously.

Coroutines are also often used for the same operations as threads; I/O operations, like fetching of data from an API, heavy CPU operations such as sorting big lists, and rendering of the graphical user interface in an application.

So what are coroutines🤨

Threads and coroutines sounds very similar to each other, so what is really the difference? Coroutines use threads, but are not threads. A thread is assigned to a coroutine to execute the content of the coroutine block. When the thread has finished executing the coroutine block, the thread is freed up, and can be assigned to some other coroutine.

The following example shows the simplest way possible to make a coroutine.

The above example first starts a coroutine by calling GlobalScope.launch {}, the coroutine lives inside the launch {} block. While the main thread continues, the coroutine pauses for one second and the resulting print is "Hello coroutine world!".

Some of the benefits of using coroutines over threads is the effectiveness and how manageable coroutines are.

So why is coroutines more effective than threads🧐

Coroutines have the ability to suspend, which means putting the coroutine on "hold". By supplying a function with the modifier suspend it can effectively be paused when for example waiting for the response of an API call.

What makes coroutines so much more effective is that while this coroutine suspends, the thread that is assigned to the coroutine is freed and returned to a common pool of vacant threads. So when the call to the API returns with a reponse, a vacant thread is again assigned to the coroutine and can then handle this response.

This way threads never uses space in memory when idle, but is utilized elsewhere while the coroutine waits to continue. Coroutines are therefore very cheap to run memorywise.

So coroutines is easy to work with?🤔

Traditionally a callback has often been used to receive a call returning from an API, like the first function in the example underneath.

vs.

When using suspend functions everything is written sequencially and becomes very easy to read and especially easy to understand what is happening in the code.

Because coroutines are written sequencially like normal code, and suspend functions are utilized instead of a potensially large chains of callbacks, coroutine code become very readable.

Coroutines intro

Man må jo få ting ut tidlig i produktutviklingen, men hvorfor vil teamet lage alt så mye bedre enn godt nok? Hva er godt nok i produktutviklingen?

Må alt være så sinnsykt perfekt?

Say you want to add a nice chart or other form of data visualization to your React app, but don't have the slightest idea of where to start. Do you write it from scratch, or do you utilize one of the many charting libraries out there? You might begin by researching d3 since it's one of the most popular data viz libraries for javascript, but you quickly get overwhelmed by the myriads of customization options and the steep learning curve. You probably don't even need all the bells and whistles that come included with it. My advice? Take a look at one of the slickest and most enjoyable charting libraries you're likely to come across this christmas - Victory.

Getting started

Victory is super easy to get started with. Start by adding it to your list of dependencies: npm i victory -S, assuming you're using npm. Now say you have some stock trading volume data that looks like this:

All you need to do to let Victory visualize this data for you is to pass it along to a Victory component with some props specifying which keys in the data objects to spread across which axes:

Great! Our chart is currently not displaying any numbers or labels, so lets fix that next. We can do this by wrapping our VictoryBar component in a VictoryChart, like so:

Alright, we're getting somewhere. Note that we're passing a domainPadding prop to our VictoryChart to stop the leftmost bar from overlapping the y-axis. The labels for the x-axis are overlapping each other, but this is easily fixed by introducing VictoryAxis and a handy prop called fixLabelOverlap:

Not bad for only five lines of code, but there's still some room for improvement.

Getting fancy

One of the really nice things about Victory is that it is highly customizable and stylable. One way to style your chart is to pass a theme prop to the VictoryChart component. You can make your own theme or use on of the pre-defined ones with VictoryTheme:

Here we're using the pre-defined theme VictoryTheme.material which renders our chart in a nice, soothing teal tint. We're also overriding the styling for the x-axis labels to make sure that our labels are easily readable and that there's some space between each.

Stock volume data is probably not best represented by a simple bar chart, which is why it's usually presented as a line chart or something similar. Luckily for us Victory makes it dead simple to change our chart type. Simply replace VictoryBar with VictoryLine and you get this:

Want to add zoom to your chart? Simply add a containerComponent prop to VictoryChart without any props, and let Victory work it's magic:

Most of the components supplied by Victory have sensible defaults and are able to render something useful without you having to do anything at all. How about adding a tooltip showing the trading volume for a particular point on the chart when hovering over it with your cursor? Easy! Switch out the zoom container with a VictoryVoronoiContainer and give it a function telling it how to display your data:

Victory is extremely customizable and tweakable while still providing a dead simple api that is easy to pick up and play with. It also provides an easy way of styling your charts by letting you define your own themes. Although Victory delivers a ton of functionality straight out of the box, you might sometimes need a little bit more direct control over the components in your chart. Which is why Victory lets you replace every Victory component with custom ones, like a scatter plot that renders cat emojis instead of boring dots:

There is still so much more that Victory can do, and since I can't fit everything into this article I urge you to check out their official documentation and try it out for yourself.

Fancy charts with Victory

Let us look at what it takes to create our own SPA

Back when I started studying IT, I wanted to create a Single Page Application(SPA). I knew some HTML, CSS, and JavaScript, and that was enough to get something up and running.

It is easy to add React, Vue, or Angular to a project that might not need it. Writing everything from scratch is a way to remind us of what these frameworks solve for us and how they limit us.

Let’s look at how we can create a simple SPA, just using vanilla JavaScript.

Templates

Let's start by creating some way of showing content. We will do this by creating a page class that will be hidden by default. In the next section, we will look at code that will add a class to the active page to show the correct content. The main page gets to be the default page shown until the JavaScript has loaded.

The templates we create for the SPA let us add static content and placeholders for the different pages. Usually, only parts of the page are dynamic.

To access the templates, you can define an object structure to make it easier to reference the correct elements.

Navigation

We now have multiple pages in our SPA, most of them hidden with CSS. To make this a SPA we need some way of navigating between the pages.

By using the function window.onhashchange, we can detect when the hash in the URL changes. Combining this with anchors, and we have a way detect when we should changing the content of the SPA.

The URL hash is easily accessible through location.hash.

To only show the active page, we need to add some styling to hide all pages that is not active.

Custom code for each page

Some of the pages need to load some additional information based on parameters in the URL. To achieve this, we can create a map of the pages with the code to run for each page.

To run the custom code, we need to hook the custom code to the navigate function by adding some code to the end of our navigate function.

Adding dynamic content

To add some dynamic content to the page, we create a new custom function for the page "toll" and add a REST call.

We use fetch to retrieve the data and reduce to generate what we will show.

Demo

Bringing it all together and we have a working SPA.

Final thoughts

Not bad, considering it is only a few lines of HTML, CSS, and Javascript? I might not use frameworks for all of my pet projects in the time to come. But I will probably continue using frameworks for more significant projects with other developers. I would prefer not to create my version of BobX.

12-Dec

Today's posts