Toasts are great, so simple but yet so useful. It is for many a part of their daily life, which is why we will today provide one of the quickest and simplest recipes for making toast. Let's dive in.

The ingredients

The following ingredients are needed:

Context API

Hooks

I would like to clarify that we are going to build a toast notification system using the context api and hooks from react just to avoid any confusion. These are relatively new tools which makes life a whole lot easier when working with both small-scale and especially large-scale code. A very brief explanation of each tool is:

Context API - A state management system that fixes many problems or annoyances that comes with handling different components state. The most basic approach for managing state is that each component uses React.setState(..) to handle its own internal state. But this can get messy real quick which is why tools such as e.g Redux are used, but these tools are not perfect either. This is where the Context Api comes in, trying to make these imperfections a bit better just to ease state management a little more.

Hooks - React hooks are basically functions that lets the user build functional components that can implement state and lifecycle features that were previously only available to components created by React.createComponent(). There are however some 'issues' with components created by React.createComponent(), e.g that its hard to reuse logic in the lifecycle functions. This is why, long story short, hooks came about.

Recipe preparation

To get started one can for example use create-react-app (CRA) to quickly get going. More information about CRA can be found in the links below. The boilerplate code that gets rendered is:

The goal is to make App() render toasts whenever needed, and for this we need two things:

A function that returns a toast.

A state where toasts can be saved and managed.

For the sake of simplicity, the function that returns a toast will be as simple as:

First we create the context:

The context works by having Providers and Consumers. The Provider shares its values with its children, which are called Consumers, without explicitly having to send them down as props. Consider the following:

The idea here is that we can wrap any component with the ToastContext.Provider and get whatever is passed through the value-prop. Neat, huh?

For the toast notification system, we need to able to store things in a state but also handle the state with different types of actions. In this case, an action can be e.g adding a notification to the state or removing one. This is where we can use the useReducer-hook.

Now we have a state manager where we can from any state either append a toast notification or remove a specific one by the 'ADD_TOAST' and the 'REMOVE_TOAST' actions. Making use of the reducer in the ToastContext.Provider gives us a working toast notification system and should look something like this:

All that's left is having some visual logic for dispatching actions and viewing the toasts.

Putting the pieces together

The last step consists of putting everything together. By using the useContext-hook in components, we make them Consumers and thus they get access to both the state of the toast notification system and to the dispatch function of the nearest Provider. The final result can look something like this:

Voila! As seen in the App()-function we do not need to send down any props to the children component, instead they can extract everything needed using the useContext-hook. This was a short introduction to hooks and the context api, and how they can be used together to create a toast notification system.

A very simple working example can be found below. For more information on this topic, please visit the given links further down. Have a good day!

A recipe for toasts

Why you, as a designer, should know the domain you are working in, not just base your choices on user insight.

In creative problem solving “a new set of eyes” and a “new perspective” are always welcome. In times of a creative drought, bringing in a person from the outside to help generate ideas can be smart. Why? Because this person has no constraints based on what is possible or not. She or he sees possibilities, not limitations, and doesn’t know the domain well enough to have that automatic filter that eliminates ideas. Using “outsiders” as kickstarters and idea generators can be helpful. However, when working on creative problem solving over time (often a designer’s job) you have to know the domain you’re in, and you have to know it well!

Domain knowledge will help you to be aware of ripples you create, prioritize correctly and help the organisation and product to evolve.

User needs are the designer’s responsibility, but user insight alone doesn’t cover the knowledge you require to accommodate these needs. You have to know the purpose of the service or system you work with, the business goals of the supplier, the supplier’s market, as well as other external factors. This knowledge will help you solve problems in a better way. Here’s why:

1 Creating Ripples\ What you do as a designer creates ripples in the system, service, or environment it is implemented in. If you don’t know the domain, you also don’t know the extent of the area you are influencing.

2 Prioritization\ Knowledge helps you prioritize correctly. What is low cost and high value? Both for the customer and the business.

3 Evolution\ Build on what you know to move the service further. It doesn’t mean you have to stay trapped in old tracks.

Creating Ripples

As a designer you are a problem solver, not a problem maker. To avoid making new problems when implementing a solution, you have to have a holistic approach in finding the solution and really know the world you are navigating in. Whether you’re an Interaction Designer solving problems in an IT-context or you are a Service Designer creating new customer journeys in a public service, your design will have ripple effects. Seeing user needs in isolation can lead you towards the best solution for the end users, but it might not be the best overall solution if the company starts losing money. Then no one is left with anything in the end.

The world is not a perfect system (as I believed as a kid). No one has that ultimate control, looking over everything to assure that the world is running smoothly. So we need people who can look up and see connections and dependencies to help new, old and changed services to run complementary side by side.

Wait what? Isn’t this service design?\ Of course this help blur the lines between design titles. Where should the Interaction Designer draw the line on what to care about in her work? Hence, the discussion on where an Interaction Designer stops and the Service Designer starts. But, as Helen discussed in the first article of this calendar, no matter what title you have, if your work includes creative problem solving you should solve problems as a part of a whole, not part by part. To do so, you have to know your domain.

Prioritization

A problem can often be divided into smaller problems. By seeing the whole picture you are more likely to understand what part should be solved first. As a designer, you should take part in the prioritizing process by fighting for the prioritization of user needs. However, if you don’t have enough knowledge about the domain you can’t prioritize correctly. Without the greater picture, you will only be able to say “this is important because...” instead of “this is more important than that, because…”

Evolution

Yes, knowing the requirements and restrictions of the industry can limit your ideas to what you already know ...oooor, combined with the right tools it can help you come up with even better ideas that advance your organisation.


"When ideas are only watered with user insights, it doesn’t give them much of a foundation to grow on. Without the fertilizer of industry knowledge, your concepts are only skin-deep and have no real impact on the core business."



- Linda C. Halvorsen, Practice Lead at Bekk


Create impact by combining user insight with domain knowledge.

There are many creative ways to unlock your brain and develop new features even though you already know the technical limitations of a system. Isn’t this exactly what a designer does? Introducing tools to help people think outside of that box? We even have a specific approach for when we don’t know the domain well enough – it’s called co-creation. So, if you don’t know your domain yet (because, of course you will right?) get the people who do to help you in the process. Broader knowledge helps the designer to create more sustainable products with real impact for both the users and the business.




.....

So, being able to facilitate workshops, utilize design methods and share user insight isn’t the only thing a designer should be able to do. You have to engage in the whole domain to be able to solve problems in a holistic and sustainable way. A designer’s job isn’t just to fight for user needs, but to do that in relation to what the organisation she works for needs, and what the world needs.

This advice might seem basic to some but there are people out there who think colorful post-its and fancy workshops are what makes you a designer (Relax, I love both!) But in my experience, these are just the tools we use, and combined with domain knowledge and user insights, we create great products and services which aim to make people's everyday life a bit better.

Thanks!\ Thanks to Anne Berit, Kjersti, Henrikke, Henrik and Sonja for valuable insights and spell check!

User insight alone won't cut it for good design

A functor might sound very strange and esoteric but chances are you have used it in some ways. Some of you probably a lot! In this article we will look at what they are and some reasons they are useful.

Lets start with a definition, and then try to make sense of it. This will not be a comprehensive explanation and we will be glossing over some details in the interest of brevity but it will hopefully be enough to you get you started.

A functor is a structure that has a mapping function that can transform the values inside the functor

As with monads a functor can be thought of as a container or structure that holds some value. But it is not just a dumb container. The structure might have different states or behavior which makes the values inside inaccessible. The mapping function will ensure that we can safely access and transform the values.

By having the structure define and uphold its own rules through the mapping function we do not need to know all the different rules and edge cases for accessing the value. The functor handles this and all we need to do is to use the mapping function. It makes our coding life easier! 😄

This might seem to make little sense now but stay with me. We will start off in a language many might be familiar with: JavaScript!

My first functor

The functor that are used the most without people even knowing it are probably arrays in JavaScript. One of the ways that you know something is a functor is if it has the map function.

The map function on arrays are used to transform (or map 😎) the values inside it to another type (or another instance of the same type):

Map takes one parameter which is a one-parameter function. When sent (or applied) to map this function will in turn be called with each of the values in the array giving us a new array containing the newly transformed values. In the above example we are transforming the numbers to strings.

Note that when mapping arrays we always get back an array. The values in it can be transformed into strings, ints, objects or arrays (or any valid JS type) but they will always be inside an Array.

The Maybe functor

In the statically typed functional language Elm these functional concepts are used all the time but without actually talking about the technical names. Lets look at one of the most used besides lists/arrays: Maybe.

Elm does not have null or undefined so values that might not be available have to be represented in other ways.

For this the Elm standard library has a type called Maybe, which is defined like this:

This defines a type called Maybe which contains a value a (small letter means any type). It represents two different "states": Just for when you have a value and Nothing when you have nothing.

You might have been in a situation where some of the data you get from the backend might not always be available so you end up having to check of fields are null or undefined. In Elm we can use Maybe to model this.

Lets say you have a field of type Int that might not be available in all situations. To model this we can use the Maybe type. This gives us a value of Maybe Int. We might later need this value to be a string. We can use Maybe.map and String.fromInt to easily convert it to Elm’s string type:

Later we can use Elm’s case expression to get the value so we can display it to the user.

More complicated structures

The Maybe type is fairly intuitive in that we understand that when we have an instance of Nothing there is nothing to do and it is relatively simple to just use a case expression to access and transform the value that way. If the type/structure we are working on is more complicated like the RemoteData type the map function makes our life easier.

This type represents and helps us model the states a HTTP request can be in. In many situations the interesting part is the Success state and its value. If we want to access the value we expect after a successful request through the case expression it would look something like this:

This gets tiresome very quickly! Functor to the rescue! 🎉

RemoteData.map makes it easy for us to transform this value without a case expression where we need to handle every possible state of the type.

This has the same functionality as the code above:

Does not that look a bit easier to use? 😄

There is a lot more to be said about functors but I hope this was a good intro. If you want to explore it more I recommend you to take a look at Functors, Applicatives, And Monads In Pictures which has great illustrations or listen to LambdaCast episode 16 on functors.

Functors - What are they?

Er det 1:1 mellom designer og design? Eller er design "motoren" mer enn det en designer utøver? Hvordan ser vi på design arbeidet opp mot evolusjon og hypotesetesting? Er design simulering? 

Linda Halvorsen, Jens Andreas Huseby og Moquan Chen kaffeprater.

Hva er forskjellen mellom design og designer?

We all know it; application security is a shared responsibility and everyone in the team should act according to the secure lifecycle development process. But our experience is that security is one of the first non-functional requirements that are dropped when deadlines approaches or when management is setting up a budget for the next period.

Some organizations have a security team or group. That can be a good start, but it can be problematic that the people that "does" security, are split from the people that "does" application development. Often the security team is busy making security policies, while the development teams are busy producing functionality. Neither meet to discuss and exchange points or view. At some point the security team may call for a meeting where they present their brand-new security policy, which they already told the board all the teams are following. The development team themselves are usually behind schedule and do not have time to implement the newly presented security policy.

How can we improve on this situation? One approach is to introduce a role in the team which is the liaison between the security team and the development team, the security champion. Why a role you may ask. In our experience managers react well to roles and by formalizing this, some actual pressure can be applied when it comes to prioritizing security related issues (before they are exploited).

The security champion is a member of the team which should help the common responsibility of improving security awareness. She should be the goto when it comes to discussing best practices and how other teams approach common issues. The security champion should be part of the prioritization of the team's backlog, in cooperation with the product owner. It is important that the product owner understands that he is responsible for all aspects of the product being created, security included.

In a multi-team environment security champions should meet regularly to discuss and coordinate security efforts across teams. The CTO and/or members of the security team should be present at these meetings. The champions should then update the team on any changes to policy or upcoming blockers from other teams. By being a single point of contact for the team, the risk of reported security issues being lost or forgotten is reduced.

The security champions can also arrange awareness activities; such as monthly presentations of best practices, lessons learnt, workshops on various tools and capture the flag events. By raising the focus on security in the whole organization, the risk of being the victim of a security incident is greatly reduced.

If you don’t have a security champion on your team, take the opportunity and become one!

Who is your security champion?

We used to have a pure Java application. Then we decided to add some Kotlin code to it, just to try it out. Now there is almost no Java code left. How did that come about? Read on!

Kotlin is a language rapidly growing in popularity, not least among the established Java communiy. One of the key benefits is that the language has full interoperability with Java and the JVM which makes introducing Kotlin to an existing Java project pretty straightforward. Without going into details all you have to do is basically make some minor changes to the configuration of your project and you will be good to go. This is probably one common way the language makes it way into the lives of Java developers.

This article is based on working with a backend application doing microservices over REST. A relatively small codebase (< 5000 LOC) made rewriting the Java portion of it a rather manageable task not requiring tons of work. We'll share some experience gained along the way, both with the language itself and using the Kotlin plugin to IntelliJ.

The Kotlin plugin to IntelliJ

Once you've started to add Kotlin code to your Java project and the share of Kotlin increases you might get to a point when the mix of the two languages starts to hurt a little bit. It can be a bit confusing to constantly switch between the two different syntaxes and you might even start to prefer doing your coding in the Kotlin portion of the source tree. This is the moment when you might decide to rewrite your existing Java code to Kotlin. This is also when the Kotlin plugin to IntelliJ comes in handy.

The plugin bundles a Java to Kotlin converter that works in two ways. You can either copy a piece of Java code and paste it into a Kotlin file and you will be prompted if you want the plugin to translate it. Alternatively, you can select one or even multiple Java files and from the code menu choose to transform them into Kotlin source files. Doing so you might end up with a piece of code that you are happy with, but often, some refinement is needed to get the code to the desired quality or even to get it to compile at all. The plugin might also suggest improvements in your code where you simply press Alt+Shift+Enter to accept them. It should be mentioned that the migration tool in the plugin is still in an experimental state, and that the behavior experienced at the time of writing might have changed by the more recent versions. Nevertheless, let's look into a few random examples.

DTOs

Working with REST, either consuming or producing services, you will often have DTOs (data transfer objects) used when serializing and deserializing data transferred as JSON. In Java you create the fields you need along with the necessary methods such as getters and setters. When converting those classes you will very likely run into the concept of immutability and nullability. Kotlin makes a very clear distinction between whether a data type is nullable or not, and the compiler will enforce how you may handle those types in your code. If the type is declared nullable then you basically have to implement checks to avoid the all too familiar NullPointerException (NPE). So when the plugin has converted your code you often (given you have not used the @NotNull annotation) end up with nullable String type properties. A nullable property is declared with a ? following the data type. This may in turn result in compilation errors elsewhere throughout your existing code where you access those properties. You might feel rather certain that these properties will never be null and be tempted to start inserting the !! operator (which converts the nullable type to a non-nullable one) and accepting the risk of a NPE. A better approach is probably to alter the class generated for you and make qualified decisions if a property in fact can be null or not. In our case that meant taking a closer look at the swagger-documentation of the services consumed and making only the mandatory fields not nullable. The others are handled properly for null values, fulfilling the intention of more robust code. Furthermore, we made use of the data class in Kotlin, which dramatically reduces a lot of boilerplate code.

So a simple Java class like this,

would finally end up with this nice little Kotlin class with immutable properties (due to the val keyword) where we are assured that non of them will never be null. In addition, data classes will have constructors, toString(), equals(), clone() and a few more standard functions created automatically for you.

String concatenation

String concatenation in Java is usually done either using the + operator or using the method concat() on the String class. There are other options as well, but let's stick to the two most basic ways as they are the most commonly used. The java code might then look like this.

Or like this:

Note that getCode() returns a primitive int and thus has to be converted into a String in order to be used with the concat method. Using the plugin we end up with the following for both cases:

As we see, the plugin seems to prefer using the + operator. The Kotlin String class has a function plus() that is very similar to Java's concat() counterpart that would have looked like this for the second example:

However, there is a third option we tend to prefer, using a feature in Kotlin called String templates. This makes for very compact and highly readable statements like this:

The concatenation itself looks fine now but while we are at it, we may improve this code further. Firstly, we can safely get rid of the question mark the plugin strangely added to the function making its return String value nullable. Furthermore, since this is a single-expression function we may also remove the curly braces and assign the return value directly to the function. Having done that we may then even remove the String return type completely, making advantage of type inference. The result is this:

It would probably have been reasonable to make this function a member of the Foo class itself, but for the sake of the example we left it like this.

Extension methods

Kotlin has a concept of extension methods where it is possible to add new functionality to existing classes without making use of inheritance. This can make the code highly readable and thus, easier to maintain. In Java, we often find ourselves passing objects as parameters to methods that perform some sort of logic based upon then. There is nothing that prohibits us from doing basically the same thing i Kotlin, and the plugin will often do just that. While the code might feel familiar from the Java version of it, you should keep en eye out for good candidates for an extension method here and there. A slightly simplified example from our case where we had the following Java code that gets a response from our REST call and we want to return a deserialized representation of the JSON response. The Response class resides in the package javax.ws.rs.core.

Converting the code to Kotlin we ended up with this:

This is very similar to the Java version and calling the function may look like this:

We then refactored the converted class and instead made the function into an extension method on the Response class. To get rid of the rather ugly class parameter we also made it an inline function with a reified type parameter. That meant we could get the class from the type parameter using T::class.java.

Now we could call our function like this instead:

Or alternatively this way:

Open classes and functions

Using annotations in your code, for instance when working with Spring applications, the compiler will often require that the annotated classes and functions are overridable. In Java that is the default and the compiler will have nothing to complain about as long as you have not explicitly declared your classes and methods as final. In Kotlin it is the other way around, where you explicitly have to make them overridable using the open keyword. This is something the plugin usually wouldn't do for you, thus giving you compiler errors you have to deal with. There are a couple of options to solve this. You could either configure your project using the openall-plugin (which will handle everything transparently in the background) or by adding the open keyword yourself. We went for the latter as we found it more readable and not introducing another dependency doing its magic, but it is really a question of taste. A class annotated with the Spring @Configure annotation would then look something like this:

Final words

OK, these were just a few simple examples, but hopefully something that gives you an idea of how to approach Kotlin from the Java world. It can be done gradually in small steps with the help of a handy plugin to speed things up. Thankfully, it is still plenty of room for creative thinking and to discover and make use of all the features and capabilities in Kotlin. It should be fun!

Going from Java to Kotlin

So like every year, Christmas is approaching too fast, and like every year I’m late at writing Christmas cards! Like all other complex problems in life, also this can be tackled by using the open source infrastructure as code tool Terraform!

Now from last year, I had the list of friends and their respective addresses in a CSV file that looks like this:

I then quickly put together a Christmas card template file that looks like the following:

The Brits are a bit peculiar about how they address mail compared to the rest of us, and I therefore had to add an if directive in the template.

In order to actually generate some Christmas cards, I decided to write all the cards to disk using the local_file resource in Terraform in combination with the new for_each resource meta-argument and the new for expression (these features were added to Terraform in version 0.12.0 and 0.12.6 respectively). Combine all that with the templatefile expression, and you end up with this Terraform "one-liner":

Whys is this cool you might ask? Well, there a lots of features in Terraform that allow you to do stuff that you can't do with any of the other infrastructure as code tools out there. Here's a brief list of features that come in handy on a regular basis when working with Terraform:

Built-in functions that let you do anything from math to crypto

Miscellaneous Providers that let you do anything from generating TLS certificates to random passwords. These are all non-cloud related resources really.

Source code

If you're keen on running this example on your laptop, do the following:

Terraforming Christmas Cards

So we've all mostly heard about GitHub, right? But did you know there are other thriving open source hosting platforms out there?

GitHub

First and foremost, how about some GitHub history. Started in 2008 as Logical Awesome LLC, it changed its name to GitHub in 2010.

It also saw a huge milestone in July that same year, when it reached one million repositories created. GitHub was bought by Microsoft in June 2018 for $7.5 billion, a highly discussed topic that not all GitHub users were happy about. Today, GitHub has more than 40 million users in total, with 44 million repositories created in just the last year

All in all, GitHub has a pretty big share of the market. But it does have some great competitors. Let's take a look at a few alternatives to GitHub.

"The others"

Let's start with Bitbucket.

Bitbucket

Bitbucket has seen a lot of success over the last couple of years and just this year it announced achieving an impressive 10 million Bitbucket Cloud users. It started as a start-up back in 2008, and was aquired by Atlassian in 2010.

In the beginning, it only offered support for Mercurial projects, but rolled out Git support in 2011. Some of the really convenient features it boasts are its integrations with other Atlassian products. To support the whole product team and the development process, one can use Jira, Trello and Confluence together and have an impressing amount of features supporting the development.

Azure DevOps

Azure DevOps (formerly Team Foundation Server (TFS) and Visual Studio Team System) was released as a platform by Microsoft in 2013 and called "Visual Studio Online". It grew to host many of the features from TFS "as-a-service" on its platform, including work item tracking, extensibility via APIs and supporting different source control systems, e.g. Team Foundation Version Control, Git and more.

Azure DevOps Services' Continuous Integration and Continuous Delivery is using open source for its pipelines. This year Microsoft also released Azure DevOps Server 2019 which can be installed on-premises.

Azure DevOps, GitHub and Bitbucket are proprietary software.

GitLab

GitLab on the other hand, is a product that both enables open source, and truly embodies its essence. The first commit was made by its founder Dmitriy back in 2011, in a house in Ukraine without running water. Gaining popularity it grew at a steady pace and became a corporation. GitLab Enterprise Edition launched in 2013, and by 2016 it had over 1000 contributers to the source code.

Hosted on its own platform, the code base is still open source and its GitLab Community Edition is distributed under the MIT Expat license.

The story of GitLab's origin to where it is today is a great success-story for its founders, its community and open source! 👏

GitHub, and what else?

Garbage collection (GC) has been one of Java's most compelling features since its inception. Allowing developers to focus on the core functionality of their application instead of memory management.

Garbage collection is the mechanism used in Java to free up unused memory. In order to achieve this it keeps track of all objects and determines which ones that safely can be removed from the heap, thus freeing up that precious memory.

When a Java process launches it allocates memory for the heap, metaspace (PermGen successor), JIT codecache, thread stacks, and shared libraries depending on how much memory is available to the process. Though the metaspace and codecache also have garbage collection we're going to focus on the Java heap.

The Java heap is divided into three separate areas as shown below;



Eden space is the part where newly created objects are placed, so whenever you create a new object it is placed in eden space. Objects in eden space that survive a garbage collection are moved to the survivor space. And if the object survives in survivor space long enough to exceed a threshold (ex. survived 8 garbage collections) it is promoted to tenured space.

This process of promoting objects through different spaces is known as generational -or ephemeral garbage collection, and is based on the hypothesis that most objects are likely to be short lived. It also is the reason why we talk about minor and major garbage collection. Typically minor GC works on the young space, which is the combination of eden space and survivor space. While major GC does its work in the tenured space.

Phases in a GC cycle

After this short introduction to Java's memory model it is time to take a closer look at what is going on during a GC cycle.

Marking

In any GC algorithm the first thing that needs to happen is the marking phase. During this phase the algorithm looks at the heap space in question and tries to figure out which objects it can remove.

Before marking:


After marking:


Marking objects as ready to be garbage collected (dead objects) can be achieved in several ways. The simplest to reason about is the well-known reference-counter approach; where each object keeps track of how many references to it are floating around. Whenever a reference is removed, the referenced object decrements its reference-counter, and when it reaches zero it can be marked as dead. This approach does however have some limitations when it comes to cyclic references, and is not used by any garbage collector in Java. Instead the Java GC algorithms uses graph traversal algorithms to find which objects it can reach.



By starting at what is referred to as GC Roots and following all references it can find, it is able to correctly mark all objects that are reachable. Unreachable objects are then marked as dead objects and can be garbage collected. What is considered a GC root may vary, but include local variables and input arguments for any currently executing methods, active threads, static fields of loaded classes and several more. For example when running a minor GC (remember, just the young space), then every reference from tenured space into young space is considered a GC root.

Sweeping

After marking all objects that can be removed from memory the GC moves on to actually freeing up the space. One approach would be to just free up all space occupied by dead objects.

Before sweep:
After sweep:


While this does free up space, but you run the risk of encountering a OutOfMemoryError later on if you try to allocate memory larger then any of the given free regions. To solve this one might turn to compacting;



Compacting, moves all object to the start of the memory region, and thus allows larger allocations to happen in the future. The downside of course is that the GC time increases as nothing is free in this world.

As an alternative to compacting the memory in place is to use seperate region and copy live objects to another region. In the heap-description above we saw that survivor space was divided into to regions; S0 and S1. One approach could therefore be to alternate between these two:

Before marking:
After marking:
After copy


Mark-and-copy is similar to mark-compact in that it also reallocates all living objects. The advantage of mark-and-copy is that since all objects are moved to a new region it is able to copying objects simultaneously with the marking phase, though at the expense of an extra memory region.

GC Algoritms

We finally arrive to the point where we can talk about the different GC algorithms.

Serial GC

Serial-GC uses the mark-copy approach for the young space, and mark-sweep-compact for tenured space. As the name somewhat implies it is a single threaded collector relying on stop-the-world (all application threads stopped) pauses to get its work done. As a result it is best suited for environments where you have a small heap size (<200Mb), and single CPU core available. Enable it by passing: -XX:+UseSerialGC when starting the Java process.

Parallel GC

Parallel-GC (also known as throughput collector) is very similar til serial-gc, but uses multiple threads during marking, compacting and copying. In Java 8 this was default algorithm for the server-class machines, while client-class machines used serial-gc. A computer is considered server-class if it has 2+ physical processors and 2+GB of physical memory. Enable it by passing: -XX:+UseParallelGC -XX:+UseParallelOldGC to use it in young space and tenured space, or -XX:+UseParNewGC -XX:+UseConcMarkSweepGC to only use it in young space (CMS in tenured space).

Concurrent Mark and Sweep (CMS)

This is the first GC algorithm which doesn't rely on stop-the-world pauses for all its work, hence the concurrent part. It uses standard parallel mark-copy for the young space, and concurrent mark-sweep in tenured space. The goal of this algorithm is to minimize the pauses due to garbage collection, and it does this by running part of the cycle concurrently with the application threads. Enable it by passing: -XX:+UseConcMarkSweepGC, it is however deprecated in Java 9 and scheduled to be removed in Java 14.

G1GC

G1GC (Garbage first GC) is the first region-based GC. Which means that the heap illustration from earlier doesn't really fit anymore.

Example of heap with multiple regions:


G1GC keep track of the amount of live vs dead objects in each region, and collects garbage in the regions that contains the most garbage, hence the name garbage first. This allows the collector to avoid collecting the entire heap at once, and instead it can select a few regions. Which in turn leads to smaller pause times needed by the GC, thus making G1GC very well suited for application where latency is important. G1GC was first introduced in Java 7 as an experimental GC, and made the default GC in Java 9. Enable it by passing: -XX:+UseG1GC.

The future

Recent releases of Java have seen the introduction to two new alternatives; ZGC and Shenandoah GC. Both being low-pause times, "fully"-concurrent, region-based and capable of handling large heap sizes.

ZGC is available in OpenJDK 11 (linux only), with macOS and windows support planned for JDK 14 according to the OpenJDK Wiki - ZGC. If you are interested in the inner workings of ZGC have a look at this post at baeldung.com.

Shenandoah GC is available upstream in OpenJDK 12 with backports to OpenJDK8u and OpenJDK11u, which should make it availble to most programmers out there. Progress can be tracked at the OpenJDK Wiki - Shenandoah GC.

Garbage collection

The most important part of any data science project is the data. It does not matter how fancy your algorithm is if your data has low quality or does not capture the relationships you are interested in. In the end it all comes down to the simple truth: “garbage in, garbage out”. Data cleaning refers to a variety of methods for improving data quality. It can be a time consuming and challenging process, but the reward for properly cleaning your data can be great.

¨




Inspect the raw data

First of all, look at your data. It may sound trivial, but it is easy to forget. Once your data is in a tabular format, use a couple of minutes to scroll through the table and look at the entries. After inspecting the table, calculate statistics (mean, median, standard deviation, etc.) and visualize the data (e.g. scatter plots, line plots, heat maps, etc). The above steps often give you a sense of the overall quality of the data, and if you know the domain you are working with you will often be able to spot obvious errors in the data.




Remove irrelevant variables

Irrelevant variables may be defined as variables we know to have nothing to do with the problem we are trying to solve. For example, say you are trying to predict the probability of a disease in a population. In this context we can be pretty sure that a persons phone number will not affect the probability. Hence, we can safely remove this variable from the data set. Another example is data sets with 100% correlated variables. If a data set has two variables that are 100% correlated they contain the same information. Hence, we can safely remove one of them - pick your favourite!




Drop duplicate observations

A data set may contain observations that are repeated by mistake. Combining data sets from different sources often results in duplicate observations. Data from a database where users may submit entries multiple times may also include duplicates. In many situations duplicated records do not represent two separate observations and we should therefore remove the extra ones.




Handle missing data

Data sets without missing values are widespread when you learn data science and machine learning in school, but very rare in the real world. Most machine learning algorithms do not work on data with missing values, hence handling missing data is an unavoidable part of being a data scientist. There are many approaches available.

Drop observations with missing values. If your data set contains relatively few observations with missing values, removing them is the easy fix.

Impute missing values. A more complex approach is to replace missing values with “best guesses”. This is called imputing missing values and there are many approaches available. One common and easy approach is to use statistical values like the mean or median for the variable in question. A second approach is to use a linear regression model between correlated variables to impute the missing values. A third is to copy values from similar observations based on a clustering algorithm. 

Now, you're all set to start cleaning for Christmas!

Who said Christmas cleaning can't be fun?

This day is going to be all about blending colors, or rather how you can do blending in CSS with mix-blend-mode! Those of you that have limited experience playing around with colors may wonder what blending is. It’s quite simple, blending is when you mix two colors together in order to get a new color - and yes, it’s exactly like when you played around with color tubes in kindergarten!

How does mix-blend-mode work?

Just like when you mix real colors on a palette, computers can do different computations on two color codes to find a new color! Some of you may have played around with this kind of blending in photo editing software where you can apply different effects on photos. CSS supports blending as well with the mix-blend-mode property, which allows us to blend the colors of different elements!

The color of the element with mix-blend-mode set will be blended with it’s background color. For example, we could put mix-blend-mode on a text-element over an image to have the text color adjust according to the color of the background, or we could blend two images together to achieve quite a cool effect.

However, blending with a computer is not quite as straightforward as when you mix two colors on a palette. There are a heap of different ways to blend colors with computers, which in CSS can be specified with different blending modes, but more on that later! Lets get on with some examples on how it actually looks like in practice!

Blending text on top of an image

Blending text with a picture is actually quite easy. All it takes is to add the mix-blend-mode property to the element that you want to blend with the background (in this case the text):

You can see the effect of this code used together with the blending mode difference in the image below! This blending mode calculates the difference between the colors being blended, for example if we would blend white (#FFFFFF) and blue (#0000FF) we would get yellow (#FFFF00)

Blending two images with each other

You can also blend images with each other to get quite a cool effect. By just playing around a little with different blend modes I ended up with a red santa above some snowy landscape - quite nice!

In this case there is a black background, then a div containing two img's, first the snowy landscape and then the santa. The order of the elements does matter, in this case the snowy landscape is first blended with the black background, and the resulting image is then blended with the red santa!

What else?

I recommend just playing around with mix-blend-mode a bit, it's quite fun and you can end up with some beautiful effects! It's also possible to have dynamic elements that move around while simultaneously blending, and in modern browsers there is support for blending SVGs as well.

Happy blending!

Blending color codes with mix-blend-modes

Years ago, the JavaScript language was gifted with the Symbol primitive. Unlike many other language features in ES6, it was not simply a shorter syntax for doing something otherwise previously possible. It is not obvious what the point of these symbols are, and they are not commonly used. If you try to read up on them, you will find that many articles focus on details that are not important to the purpose of symbols. This article will explain what they are, what the point is, and give some real life use cases.

What is it?

Symbols are a type of value in JavaScript, like numbers and strings. It is a core primitive, not a high level class deriving from Object. Let's create some symbols to see how they work:

Yes, but what is the point?

Given the previous description, symbols seem like weird strings that you cannot see. But their main purpose is that they can be used as property keys on objects.

Property keys in JavaScript have before this been confined to strings:

But now, these keys can be symbols:

And these symbol keys are treated separately from the old fashioned string keys:

I still don't see the point!

Yes, of course you don't. Why write 3 lines of code when 1 would suffice? Their purpose come to light when you consider that all symbols are unique and will never have a name clash.

I am so random

Symbols are the elegant solution when you otherwise would make up some random value to separate things. A typical example is handling the arrow keys on the keyboard:

Now consider what happens if we pass the wrong value to this function:

The messy solution to this might be to make up some very random values for arrowkeys. But the elegant solution is to use symbols instead:

What if JavaScript itself wants to extend you?

The non-invasiveness of symbol keys is used by the JavaScript runtime itself to make it possible for us to override standard features like conversion to primitive values:

Imagine if, instead of symbols, there were reserved function names on Object to override for these kind of extensions. That would make name collisions much more frequent.

Even for ... of loops can have their behaviour defined:

Conclusion

Although a small feature in JavaScript, symbols bring something to the table that no other mechanism can. Use them to avoid polluting the namespace, and when you need unique values without being creative. Use it wisely, and happy coding!

20-Dec

Today's posts