If you're looking for the definite answer to how you should structure you React apps, this article probably isn't for you. If you're interested in seeing how it can be done - read on!

How to structure files and components in a project - or even components in themselves - is always the root of never-ending discussions. And like a lot of topics, there is a lot of strongly held opinions and different ways to solve it.

Your own opinion is probably influenced by your personal preferences on what's most readable, easy to parse, or just nice to look at. When you're a part of a team, you'll typically need to agree on these structural decisions, which isn't an easy feat in the best of teams.

These kinds of discussions can be compared to two people arguing which of their favorite colors - blue and red - that's the best one. They'll always voice their opinions, state their case, and then just keep on having the same opinion. They'll never agree on purple, to put it that way.

So I'm not hoping for agreement on this. In this article I will show what preferences I have when it comes to structuring my React code, and why I have those preferences. Hopefully you'll learn a thing or two, or at least gain understanding for a different point of view.

Puzzles

First, a few words about how I am as a developer. When I code, I see what I'm building as a building blocks - like a puzzle. Every thing I make, every function I write or component I implement, is a part of a larger picture.

I'm also very fond of visualizing what I work with. I tend to draw when I explain, and I love when what my users are using, the actual pieces of UI, is reflected in the code base as well.

Another thing that's important to me is to get a top level overview quickly, and understanding what a component does as simply as possible. Both through spending time choosing the right names, but also through visualizing hierarchies through usage.

Santa's gone digital

Let's look at a fictive project I've created for this article - I've called it Santa's digital wish list. Everybody can visit, create a user, add their wishes and see their profile page.

Below, you'll see the file and component EditMyInformationToSanta, which really needs a refactor. It's the component that lets us add all the info that Santa needs in order to provide you with the correct gift for Christmas.

It's a lot - so let's just jump in.

The component itself isn't really very advanced, but since it's a pretty big form with a lot of information, the file size (or rather, the file length) becomes pretty huge. To improve this, I'd start splitting the code into components.

Some of the most obvious culprits are the repeating inputs and radio button layouts, so let's start with those. In the example code below, you'll see what I'm left with after the refactor:

Note that, even though I'm not sure if the inputs and radio buttons will ever be used by other parts of my application, I'm still choosing to move them into their own separate files. These are placed as close to the original component as possible, so they're as close as possible to where they're used.

Keep your components separated!

Many developers prefer to keep local components like these in the same file as they're used, but I prefer a maximum of a single component in each file. To me it's much quicker to gain that top-level overview by visiting many files with a single component in each, instead of a single file with many components.

In this case, the difference probably wouldn't be huge, since the components I refactored out isn't very big, but the problem starts to surface once they start to grow. In a few weeks, you'll end up having to scroll up and down the file to find the "root" component, and it'll be a huge mess.

To skip internal discussions about when to do this refactor, or how big these extra components need to be before we pull them out into their own file, I just prefer to implement this simple rule.

Extract by domain

Back to our example. Even though we've extracted repeated markup into reusable components, I'm still not completely done with my refactoring efforts. I want to take it to the next level, and refactor out code that are grouped by the task they're representing. In the following example, I've extracted components for each section of the form - AboutMe, LetterToSanta and MyWishes.

You can see the complete code for this project in its Github repository.

Now we're left with a directory that contains the file EditMyInformationToSanta.jsx and a bunch of simple component files. Each file is small, easy to reason about in isolation.

The section components are very local to this file, so they reside in the same folder. The reason I refactored this code out in the first place is so the main file is visually nicer to look at, and easier to understand. We'll can also place other relevant files here, like styling, images, texts, utilities or other resources.

Domain vs Components

When your app grows, like when we add live-tracking of Santa's movements using the SantaLocation API, or the new top 10 wished gifts feature, you'll eventually start running into places where presentational components can be reused. To separate these from the rest, I like to split my code into two folders - /components and /domain.

/domain contains the domain logic of my application. It's where EditMyInformationToSanta lives, and all of its buddies.

/components is where I place components that are reusable and can be used across the project. TextInputWithLabel is a typical candidate for this kind of reuse, and will be moved from the /domain/edit-my-information-to-santa folder to its own folder in /components/text-input-with-label whenever that need arises. Like in the /domain folder, this one might also contain relevant files as texts or styling.

Keeping it clean

We've spent the last couple of minutes stepping through the process of how I like to structure my React code. To summarize:

I like to extract presentational logic into their own components, both where it makes sense in terms of reuse, and where it makes sense in order to simplify the root component.

I extract each component into its own file, and make sure to keep to that rule as my user interface grows in complexity. I do this because I think it's easier to see what's happening in each file when there's only a single component in there. The files are shorter, which gives me that top level overview, and good component (and file) names help with understanding the underlying responsibilities of each.

Splitting the components into one-off domain logic and reusable components makes our code base easier to navigate, and it definitely makes it easier to delete dead code.

When working as a consultant, you end up spending a lot of time looking at other people's code. Making sure that process is as smooth as possible is paramount. Most of the time I alter or extend already written code, and having a simple, yet robust file structure that's easy to reason about is a great help in getting stuff done.

Agreeing on the impossible

I want to end on the same note as I started the entire article. Agreeing on how to structure you code with a team of diverse people with diverse opinions and preferences isn't necessarily easy. But do we really need to agree? I'd say yes. Not about what's the prettiest, but in what kind of code style and structure you want in your project - even if some people don't agree completely.

Getting your whole team to agree on what's the "cleanest" or "most readable" is not a realistic goal. What becomes the most important is to have everybody voice their opinions, try to empathize with why they want one thing or the other, and then try to have a conversation about what choices you should make as a team. It will definitely be a lot of give and take for most (if not all) members of your team, but I stand by that it's better to agree on a single code style and structure, instead of having a mix and match of everybody's personal preferences.

No matter what you end up with, agreeing on one solution will be the key to a neat, clean, understandable and standardized code style in your project.

How I structure my React code

Any student of computer science has encountered the concept of code as data. Most famously, in the the Lisp language, all code can easily be manipulated by code written in Lisp, dissolving the barrier between the program and its input.

And while it is less obvious, this concept of code as data can also be applied to any Java application. Admittedly, in a less obvious manner: when compiling Java sources, the Java compiler generates .class files which represent the source in a binary format. And by thinking of binaries as arrays of bytes, it is of course possible to serve those arrays as input to the very same Java application those arrays represent. Fortunately, the Java class file format is specified in the minutest detail what makes processing classes a very feasible task.

Why is this useful?

At first glance, such metaprogramming might seem overly complicated. Isn’t Java Turing complete what makes such expenditures unnecessary? The truth is that almost any Java enterprise application uses code manipulation, for convenience and to create concise APIs that base on existing types. As an example, consider that you needed to implement a security library, where any method annotated with @Secured(role = "admin") should only be invoked if the current user is indeed equipped with administrator rights. The most straightforward and reliable option would of course be to change all application code to enforce such state by adding a corresponding snippet of code at the beginning of any annotated method.

Code manipulation with Byte Buddy

Byte Buddy is a library for manipulating and generating Java class files within a running application, a tool that hides the gory details of byte array processing to make metaprogramming in Java approachable. The library’s API intends to resemble the Java programming language as much as possible to give a familiar start to developers who already program in Java.

To understand what Byte Buddy can be used for, consider the previous example of an annotation-driven security library. Using Byte Buddy, it is simple to implement a security check at runtime by for example creating a subclass of an existing class. By overriding methods of the base class, Byte Buddy can add additional behavior in this subclass and only invoke the original method based on conditions provided by the library’s user.

As an example, consider a service:

Using Byte Buddy, it is now possible to apply code manipulation to invoke the following security interceptor prior to invoking the the annotated method. Doing so, one can check if the logged-in user really has the necessary privilege:

Using this interceptor that checks any method’s annotated role against some static user management utility, a secured subclass is created as follows:

As the code hopefully indicates, Byte Buddy will override any method that is annotated with Secured and invoke the above interceptor before calling the original method. Only if an exception is thrown in the interceptor, the call to the original method will never be applied, thus securing the method call.

This is similar to how frameworks such as Spring implement annotation processing on their bean instances where annotations are processed and enforced by proxy subclasses in a similar manner. Similarly, frameworks like Hibernate, Mockito, AssertJ and many others also use subclasses to inject their code into methods where deemed necessary.

Changing code during build time Many frameworks choose the generation of subclasses for code manipulation because of convenience. Subclass generation does not require any particular setup. However, there is more to code generation. For a security framework such as that proposed in this article, it would be more reliable to add the security check directly to the methods in question. For this purpose, Byte Buddy offers plugins for Maven, for Gradle and for the command line that manipulate classes during a build. To implement such build-time manipulation, Byte Buddy offers delegation to a user-implemented interface that determines what classes are picked for manipulation and what manipulation is applied:

The above plugin first chooses classes to manipulate where classes are selected if they declare at least one method that is annotated with Secured. For those classes, it applies the same interception as before, even using the same API. After applying this build plugin, all classes with the annotation implement the security check that is now reliably executed upon any invocation and not only if guarded by a subclass proxy.

As an example, the Hibernate framework allows for build-time manipulation where the framework even generates more efficient proxies than it would by using subclasses. For convenience, the framework does however give developers a choice and applies runtime subclass generation only if it detects that its build plugin was not used.

Changing code during runtime

Finally, the JVM also allows runtime manipulation of classes by installing a so-called Java agent into a running virtual machine process. Java agents are shipped within separate jar files where their manifest points to an entry point, similar to Java’s main method, only that it is executed prior to the actual program as a premain method.

Besides running before the actual program, Java agents are supplied with an API as an argument that allows to hook into the JVM’s class loading process, giving Java agents the ability to transform class files before they are loaded. And also for this use case, Byte Buddy includes an API to make use of agent-based code manipulation as simple as possible:

As a result, all classes within any application in which the Java agent was included will apply the security check, just as if the previous build plugin would have been applied.

In the end, the right approach to code manipulation depends on the actual requirements but Byte Buddy tries to make it easy to share as much code between the different ways of manipulating Java classes. Also, there is much more to Byte Buddy and to Java agents what is too much to cover in a single blog posting but which are covered in various other blog postings and conference presentations, some of which are linked in the references below.

This post was written by Rafael Winterhalter from Scienta. Thank you Rafael!

Code as data, Java edition

The Cloud has changed the way we work. From enabling truly global scale through faster and repeatable deployments to a whole new DevEx for working with back-end services.

Case-in-point; the humble API:

An application programming interface (API) is an interface or communication protocol between different parts of a computer program intended to simplify the implementation and maintenance of software.

Software is part engineering, part craftsmanship and part fashion. The pendulum of API formality moves back and forth as we developers learn and unlearn the lessons of our forebears. From the XML WSDL definitions from the days of yore to the free-flowing JSON of REST-based systems, back to the specific protobuf definitions of GRPC, or the always backward-compatible schemas and types of GraphQL.

APIs are an important part of almost any modern software system and in apps, they are the contract of communication between the backend and the user-facing parts.

This communication is not only for the benefits of machines but is the understanding of the domain shared between the developers of the different parts of the system. And this understanding evolves over time. How do we iterate and foster creative and responsible cooperation between frontend and backend while keeping the system running at all times, even before the system is ready for launch?

Enter cloud functions and proxies!

AWS Lambda, Google Cloud Functions or Azure Functions are all excellent choices for cloud functions. That is, event-driven serverless compute platforms where you pay for what you use.

Perfect for a variety of tasks, but also useful for mocking our API surface from the start of the project, backfilling with actual services, serverless or otherwise, as they are completed. These APIs make the shared understanding of the domain explicit and they can be continuously iterated upon as more and more knowledge is gained throughout the project. The frontend guys and gals can start with any part of the app and the backend people can do the same. Thus enabling them to learn quicker and together maintain their shared understanding as running code, not in definition files or documentation.

In the example below we use Azure Functions to first create a proxy with only mock data, then use a real backend service to proxy a finished service.

Create a new Function App using the Azure Portal:

Configure your new Function App:

Function apps can be actual running code, using Functions in the screenshot below, but we'll use Proxies to create a new proxy:

In proxies, we can just return data for the given endpoint and HTTP-method. In the example below, Christmas has not yet arrived and this is the simplest service known to man:

The data need not be so trivial and the API surface will most likely be less anemic in a real example, but this is literally all that is needed to mock whatever HTTP-based service you want using Azure Functions. 2 minutes, and you have a service up and running!

Accessing the https://is-it-christmas.azurewebsites.net/ChristmasEve endpoint will yield the sad truth regarding Christmas.

Any other endpoint will reach the actional function and just show the standard template:

Now, to use an existing service and the function as an actual proxy, just use a Backend URL, for instance, a popular holiday API:

Visiting the endpoint returns the data from the proxied service:

This post has just scratched the surface of what's possible using functions to speed up API development. Don't be afraid to use the cloud just to speed up your development iterations. Even if your production environment is still stuck on-prem and Christmas is still a couple of days away, the real key to effective and efficient software development is learning.

Better API development using Azure Functions

In this blog post I will go through some practical examples of using the Try monad for exception handling in Java. The purpose is to see how we can write more functional code in a non-functional but widely adopted language such as Java.

What is a Try anyway?

The Try data structure is a special case of the Either data structure, where one side is always an Error or an Exception. The point of using a Try instead of an Either is that it becomes very clear that it contains an Error, and may contain methods such as mapFailure and mapResult.

In Java, a method returning a Try might look like the following

Here, the Try contains either a DatabaseObject or an IOException. The result therefore contains either the result OR an exception – both cases will be handled in the same code path.

A function which throws an exception would look like this.

Now, they might not look that much different. With Try, you have to treat the data structure returned with a result containing the Exception. In the other, since IOException is checked exception, you must also handle the IOException as a caller. We will see later that there are other advantages to using a Try instead of an Exception.

Often in Java we do not like checked exceptions because we do not know what to do with them and we instead wrap them in a RuntimeException. The rationale behind this is that since we do not know what to do with the IOException, it is just as easy to throw an RuntimeException and let it bubble all the way up. As an example, we might have done something like this to rethrow a checked exception as an unchecked exception:

One problem with this approach is that we can no longer know if this method throws an exception or not by looking at the signature. We can jokingly say that our method signature is lying to us.

Program flow with and without Try

To help understand the differences between using Try or normal exceptions, let's see an example. In this example we have an API-controller which receives an HTTP-request, and parses it. It then sends the parsed request down to a validator component which validates the request. The request is then handled to a service layer which in turn uses a repository. The repository can for example be an external service or a database. The response from the repository is then returned up the layers and finally to the API-controller. This path is shown in the figure below

Now lets consider the error case where the repository code encounters an error and throws an Exception. The code flow would then look like this.

In this case which can be quite common in Java we throw an Exception and it will bubble all the way up and be handled by our global exception handler which will log the error for debugging, and return a nice response to the user.

The last example will show the code flow when wrapping the result from the repository in a Try

Here we get either the result or the exception. They follow the same path back to the API-controller and will likely be handled by the same API-controller and not by a global exception handler. One advantage of this approach is that errors follow the same code flow as normal application logic.

Try in Java

There is no native Try in java, so we have to get an implementation from a library or write one ourselves. It can be a fun exercise to write our own Try-monad, but for now I will only reference some other popular libraries. Vavr contains a Try monadic container and many other functional data structures and utilities. It is quite popular and widely used. Another great, but in my experience not so widely adopted library is cyclops. Their documentation is still a bit lacking, but they do provide an excellent blog post series which goes into great depths on how to handle exceptions dysfunctional programming in java: No Exceptions.

In the examples in this post I will use cyclops Try data structure, but is can easily be replaced by others.

GraphQL and Java

GraphQL is a new way of building APIs that is quite different from traditional REST. This blog post will not go into depths about GraphQL, for that see the link mentioned above. Since GraphQL is transport agnostic, it does not use HTTP error codes to represent errors. Errors are a part of the response and not something special that is handled in some other way. This blog post won't explain the details of GraphQL, but here is how the responses from a GraphQL API looks like. It is just JSON but following a specification.

In this response both data and errors are returned in the same response. This does look a lot like our Try data structure and this is one of the places where this data structure might be beneficial.

GraphQL is also often used to fetch data from many different sources. These sources can be different databases, search engines or other APIs. If one of these sources is down we can still return a partial response with the data instead of failing the whole request.

GraphQL-java

GraphQL-java is a Java implementation of GraphQL. It implements the GraphQL specification in Java. It is typical to expose a GraphQL API over HTTP and there are many project which connects GraphQL-java to a servlet. The example code provided here will use the graphql-spring-boot package and graphql-java-tools to make it easy to expose a GraphQL-API over HTTP, provide an interactive GraphiQL-client and use the schema first approach from graphql-java-tools.

GraphQL APIs are naturally a Graph and they can be written in their own graphql schema language. A schema consists of types and their relationships with each other. A simple example of a schema with a Person type can be this.

Here we have a person type which has an id, a name and a list of programming languages. The ProgrammingLanguage type has a name and a boolean, representing if it is a functional language or not. The exclamation mark means the fields are non-nullable i.e cannot be null.

graphql-java-tools

When using graphql-java-tools we write resolvers for our fields in the graphQL-schema. These resolvers are automatically called by graphql-java-tools when graphql should fetch data for a field. This avoids much boiler plate code with plain graphql-java. A resolver can return a pojo containing the fields we should resolve and may also contain other fields which will be ignored. It can also return a wrapped pojo in either Future or CompleteableFuture. The latter one can be used with BatchLoaders which can batch together many fields from different places in the Graphql query and does a batch fetching for those fields.

An example of a GraphQL resolver for the Person field.

Using Try with GraphQL-java

Now, how do we use a Try data structure with GraphQL-java and Resolvers? The trick is that a resolver implementation can in addition to returning pojos representing the data, also return a DataFetcherResult<T> containing the datatype or errors for the field. If we already have returned a Try<T, IOException> from our repository we only need to make a generic method to fold the try from a Try<T, IOException> to the DataFetcherResult<T>.

By doing this we can easily wrap all of our exceptions all the way from the repository and up to the GraphQL-layer in our application. This allows us to write code that does not throw exceptions but instead wraps them in Try data structures and later in DataFetcherResult before returning them in the GraphQL-api.

What advantages are there to using Try instead of exceptions?

We are forced by the compiler to handle the exceptions early and cannot write code that just ignores the possibility that there might be thrown exceptions. The control flow of the application is the same and since a GraphQL response might contain both data and errors, we can anticipate what the final response will be when errors are thrown. If we instead used Exceptions it can be hard to reason about how the final GraphQL response to the user will look like.

Simply coding style when using simple resolvers.

Writing code that does not throw exceptions might be a goal in itself. It makes the signatures of our methods honest. They no longer throw RuntimeExceptions you might not notice. On the flip side the code is now filled with Try everywhere, which might be problematic since it is often easier to just throw a RuntimeException and have it handled automatically by some root exception handler rendering an error in the GraphQL-field, after stripping away stack traces of course. I think these two styles boils down to personal preferences – the end result in the API might be the same.

But java can still throw errors

Yes, the JVM can still produce OutOfMemoryExceptions and the like. There might be hidden RuntimeExceptions from external libraries being thrown which we do not know where to catch in our code.I would argue that these exceptions are fine, since when they are severe enough. Like an OutOfMemoryException there is not much we can do in code other than to crash. If you have external dependencies which throws RunTimeExceptions you can either use the Try-monad as best you can and encapsulate them or just let them bubble to the top and be handled by your global exception handler. Since this is Java you should probably have a global exception handler anyway.

In the end the answer is it depends

Should you embrace writing functional code in a non-functional language like java? Are the functional paradigm really that much better than traditional object oriented code? The answer is off course it depends. How familiar you are with functional languages and concepts? How is the rest of the code base structured? Are you starting a new project, or is this a huge existing code base which has sophisticated error handling in place already? Introducing something completely different can make the code base as a whole harder to read since it now has two different ways of dealing with exceptions. For me personally writing more functional java code has been quite fun and a big learning experience. For our GraphQL-API using the Try-monad for exception handling, it has been much easier for us to have control over the response returned to the user and the tests are easier to write since an exception is just like a normal response.

Using the Try Monad for exception handling in GraphQL-java

The seed vault just got a new neighbour - Open source is moving in and plans on staying for the long haul

Svalbard Global Seed Vault

The Svalbard Global Seed Vault is a vault for storing seeds for an extended amount of time, a sort of genebank "cold storage" (pun intended) backup for the world's agricultural history in case seeds gets lost for various reasons (accidents, conflict, equipment failures, global crises etc).

It's located on the Norwegian archipelago of Svalbard and is funded by Norway and the Crop Trust. The Crop Trust is funded by organisations such as the Bill & Melinda Gates Foundation and other governments.

From its Wikipedia page:

At this time, the total number of samples held at the vault was 967,216,[21] representing over 13,000 years of agricultural history.[22]

An impressive feat and the deposits keep coming. Deposits are also soon coming in other forms, such as open source code...

Introducing the Artic Code Vault

On the 2nd of Febuary 2020 every public repository on GitHub will be snapshotted and stored for at least 1000 years! So if you have a few gremlins in your code, now's the time to start tidying up 😜

Stored on doomsday-surviving microfilm, the code will represent the product of decades of industry innovation and software, and it will be saved as part of our sivilization's legacy for a long, long time.

As a contributor on GitHub and industry member, it feels very exciting to be part of this snapshot and history!

Read the full article over at Bloomberg for their complete coverage and interview with GitHub's CEO Nat Friedman.

Open source + Svalbard Global Seed Vault: Artic Code Vault

Innsikt er viktig uansett målgruppe, men hvor enkelt er det egentlig for oss «gæmliser» å forstå unges digitale vaner? Vi er innom YOLO, FOMO og FOLO, og slår et slag for involvering av de unge i produkt- og tjenesteutvikling. Med Vikki Walle-Hansen, Thea Snilsberg Søfting og Pia Solheim.

Hvor enkelt er det egentlig for oss gæmliser å forstå unges digitale vaner?

When knitting met machine learning, an untold love story.

You heard it here first: machine learning and knitting are officially in a relationship! Yes, knitting, the act of transforming yarn into textile and fabric.

Thanks to a group of researchers at the Massachusetts Institute of Technology, there now exists software which when fed a picture of a garnet, produces instructions for that exact pattern, used by automated knitting machines – they call it Inverse knitting. While on the subject, they are also the brains behind the project Knitting skeletons, a software which lets you design your own knitwear, without any knitting or coding experience required. Amazing!

Let's take a look at Inverse knitting!

Today's standard is using advanced knitting machines for mass production of garnets and textiles. However, these require pre designed recipes, and are maintained by low-level programming code, hence requiring a certain expertise to develop and maintain. This was the motivation behind the development of Neural Inverse Knitting; the intersection between machine learning and knitting.

So how does one combine machine learning and knitting? You might consider this an unlikely combination, however, there is a logical approach behind the problem. In essence, this is a problem within pattern recognition - a common problem in the world of machine learning.

The solution presented in the article employs methods such as generative adversarial networks (GAN), and convolutional neural networks (CNN) to automate the generation of knitting patterns for industrial use. In particular, GAN was applied to generate artificial textile samples used for training the network, whereas CNN is used to generate the main image-to-recipe-model, named Img2Prog. We will proceed by looking at the steps to solving this unconventional combination of fields.

First, a knitting language is defined, describing knitting patterns. This knitting language consists of 17 operations, and is possibly recognisable by knitting enthusiasts. Here we find operations such as knit, purl, tuck, and miss to name a few. The rest of the operations are listed in the figure above.

Second, we need training data. For this particular case, the training data consisted of several pictures of patterned clothing. However, after collecting over 2,000 real photographs of patterns from clothing and textiles, the authors realised they had to resort to artificial data. In combination with real data, artificial data plays a strong role in data science and machine learning, and is both quick and inexpensive to produce. To overcome the current challenge, the real data was melded with the synthetic images, to create regularised synthetic images of knitting patterns, using GAN. This way there was a consistency between the real and artificial data.

Finally, the model, a prediction network, was trained using CNN. For reference, the entire process is visualised in the figure below. The resulting predictor, named Img2Prog, could now be fed an image of a pattern as input, producing the recipe for the corresponding pattern as output. Amazing!

If you still have some Christmas gifts left to buy/make, you now know what to do. Find your favourite theme, take a picture, apply Img2Prog and start knitting! Merry Knitmas :D

Knitting + Machine learning = ❤️

Wonder why a MutableList<Cat> isn't a subtype of MutableList<Animal>? Ever seen the "in" and "out" modifiers in Kotlin and wondered what they do? Let's find out!

So the other day I was writing a program for my new zoo. What? Okay fine I haven't actually opened a zoo, I was writing some very boring domain-specific software at work. But it's Christmas, let's just pretend.

There I was modelling my new zoo in Kotlin. I had decided on a simple inheritance; all animals of the zoo would inherit from the superclass "Animal", and would be contained in a list of animals. Currently all I had for my zoo were my two cats; Sir Toby and Mr Winterbottom. It looked something like this:

All was good and i kept on adding new features to my zoo. Until at one point I realized something quite obvious: I need to be able to expand my zoo! Well that's a simple refactoring, I just take that immutable List of Animals and turn it into a MutableList of...

Wait, what? What do you mean my cats arent animals? And why don't I get the error when I have a normal immutable List? Confused? Try it yourself:

Now some of the more experienced of you might already be mumbling about "covariance, contravariance, decleration-site versus use-site variance, ArrayStoreException" and other technical mumbo jumbo. But for those of you who - like me - are stumped by why the mutability of my list seemingly defines whether Mr Winterbottom is an animal or not (he most assuredly is!): I'll try to explain.

Like any good developer I undid and redid my change from List to MutableList about seven times before admitting it wasn't IntelliJ trying to mess with me. I then proceeded to take a look at the signature of the two interfaces:

Pretty much what I expected except one little thing: before the generic "E" of List, there was the keyword "out". This was missing in MutableList. Huh.

A little recap

Now before we go any further, let's do a very quick recap of inheritance and generics. In Kotlin as with many programming languages we are allowed to assign a subclass to a variable with the type of one of its superclasses:

Now, of course we cannot handle this variable as if it was a cat since we've only declared it as an Animal:

This is one of the central mechanics in object oriented programming.

Now with generics, things get a little trickier. Let's take our zoo. Eventually I need somewhere for my animals to stay, so I'll make an Enclossure class. (Cage sounds so...cramped). And I'll make it generic on any subtype of animal:

And give Mr Winterbottom a new home.

So this is all well and good. We've made a generic class that can be used with any animal that inherits from the Animal-class, like a dog or a sloth. It even has the feed()-function that acts equally on all animals. However, say I now wanted to add the feature of feeding any animal inside an Enclosure:

And using it would look something like this:

Now intuitively, this might look good. However, this doesn't actually compile:

You see, in Kotlin, generic types are by default invariant. This means that Enclosure<Cat> is not a subtype of Enclosure<Animal>, and cannot be assigned to it. We say that Enclosure<T> is invariant in T. "But wait!", you might be saying... "Didn't you assign a List<Cat> to a List<Animal> earlier?" I did indeed, and herein lies the magic. If we take one more look at the documentation for List and MutableList, right underneath the signature, by the parameter descriptions it says:

interface List<out E> : Collection<E>\ E - the type of elements contained in the list. The list is covariant on its element type.



interface MutableList<E> : List<E>, MutableCollection<E>\ E - the type of elements contained in the list. The mutable list is invariant on its element type.

So what it means is that List<Cat> is a subtype of List<Animal>, since it is covariant on the generic type. Whilst MutableList<Cat> is not a subtype of MutableList<Animal> since it is invariant. Wait, so why did the Kotlin standard library even put such a restriction on MutableList? Well imagine for a moment that MutableList is covariant:

I assign my MutableList<Cat> to a variable of type MutableList<Animal>. All is fine and dandy. Until the "new guy" (a dog person...) tries to make this function:

There is nothing wrong with it, a list of animals should definitely be able to hold a Dog, which also inherits from Animal. However, what if I now run the function with my mutable list of cats? (Which we are pretending are a subclass of MutableList<Animal>)

Take a moment before you read on to ask yourself why the Kotlin people would not allow this. Why isn't the class MutableList covariant? What error will this cause? Don't worry, take your time. This whole generic business is a bit to wrap our head around.

Ready for the answer? All right. The error we'd encounter is that we are now adding a Dog to a variable defined only to hold Cats. The variable myCats originally wasn't a MutableList<Animal> but a MutableList<Cat>, one which now suddenly contains a Dog! In Java, we can encounter this error with Arrays since Arrays are covariant on its type. It's called an ArrayStoreException, and is a runtime error. Meaning nothing warns us that it will happen. However in Kotlin, they've tried to make this type of error difficult by making all generics by default invariant. So why then is an immutable List covariant? Well first of all, if we had a List<Animal> as a parameter to our addADog(), we wouldn't be able to add a dog. Since.. well the List interface doesn't have add(). So what that means, is that you can't mess with the content of List. Meaning it's safe to assign a List<Cat> to List<Animal>, because none can make the mistake of adding a Dog or a Sloth.

Can we generalize this rule to any kind of generic class? Well, kind of. The rule as far as I've understood it is this:

Only a class whose members strictly return the type on which it is generic can be covariant on said type. Meaning no functions of the class can take/consume a parameter of the type, they can only return it. This in turn means there is no way for such a class to have a function that can take a "wrong" subclass and add it to its member variables. We call classes that only return their type producers. Producers are covariant on their type. List<E> is a producer, and is therefore covariant on E. However MutableList has the function add(element: E), this means the class doesn't strictly return, it also consumes a parameter of the type and thus is not a producer, nor is it covariant on the type.

Always an exception

Now before some of you get all up in arms about the List-interface having functions such as contains(element: E), which quite clearly consumes a parameter of the type, and therefore should make List not strictly a producer. Let's take a look at the List implementation:

Notice the @UnsafeVariance annotation? Yep. I just gave you a strict rule to follow, and one minute later I show you a exception to it. As long as you annotate the functions of your producer that do consume its type with @UnsafeVariance, Kotlin won't complain. This means Kotlin forces you to make a conscious decision to break this rule, if you so wish. At least it tried.

Out

So how do we annotate a type in such a way that our generic class will be covariant on it? Some of you might already have guessed it: the out modifier. Doing so not only makes our class covariant on this type, it also stops us from making any function that take this type in as a parameter, and only allows us to return it. In other words: the type can only go out of our class. Hence the name! This is also why MutableList does not have out on it's type, since it has functions that consume it without the annotation.

More cats!

Phew! Still hanging on? All right we have one more thing. Remember that Enclosure-class?

Now, this clearly clearly breaks with our rule: it has the function moveInAnimal(animal: T) that consumes the type and is therefore not strictly a producer, and cannot have the out-modifier on T. However it does not have any members that return the class, meaning it is strictly a consumer. Perhaps there is a similar rule for that?

Let's say for a moment I want to add this function:

And it works splendidly!

Contra- what?

But what if I have an Enclosure<Cat>?

Wait what? I can't add a Cat to a cat house? It's because we made the parameter Enclosure<Animal>, and since Enclosure isn't covariant on Animal, Enclosure<Cat> is not a subtype of Enclosure<Animal>. Nor can we make it covariant, since it has a consuming member function. What I really want is to have a function that accepts both Enclosure<Cat> and Enclosure<Animal>. What if I told you there was another way? Imagine if we could say that Enclosure<Animal> was a subtype of Enclosure<Cat>. Yes you read that right, the Enclosure with the supertype Animal is a subtype of an Enclosure<Cat>. This is called contravariance. The opposite of covariance! This way we could write the function:

And it would accept both Enclosure<Animal> and Enclosure<Cat> as parameters if Enclosure is contravariant on its type. Now before I give you the secret, let us look at what problems might arise from allowing contravariance. Imagine for a moment that the "new guy" added a function to get the inhabitant out of an Enclosure:

So that he could play with the kitten before moving the new inhabitant in:

Now if Enclosure is contravariant, this means that one could call this function with an Enclosure<Animal>-parameter:

This looks fine. Until we do something like this:

What happens inside our moveInCat() function now? As long as Enclosure is contravariant, getInhabitant() might return a Dog. And dogs don't purr! You'd get a class cast exception.

In

This is a very similar problem to our covariance problem. So the Kotlin people made another rule: If you want contravariance your class must be strictly a consumer. This means that there is no way for the class to return the underlying type, only take it in as a parameter. In our case, it would make the function getInhabitant():T create a compile error. So let's remove that function and add the - you guessed it - in modifier to T:

Our Enclosure class is now strictly a consumer. And with the addition of the in modifier it is now also contravariant on T. This means we can now do:

and use the function like this:

TL;DR:

A generic class is covariant on its type if classes containing subtypes of its type are also subclasses.

A generic class is contravariant on its type if classes containing supertypes of its type are also subclasses.

To make a class covariant you need to add the out modifier to its type, and ensure that no functions consume the type.

To make a class contravariant you need to add the in modifier to its type, and ensure that no function returns the type.

The ins and outs of Kotlin

A cellular automaton is a system consisting of cells of numerical values on a grid, together with a rule that decides the behaviour of these cells. By applying the rule repeatedly on each cell in the grid while visualising the grid in some way or another, one often gets the effect of some evolving organism with complex and intricate behavior, even from relatively simple rules.

Cellular automata comes in many shapes, forms and dimensions. Perhaps the most famous cellular automaton is Conway's Game of Life (GOL). It consists of a two-dimensional grid where each cell contain a boolean value (dead or alive). The accompanying rule decides whether or not a cell should be dead or alive based on that cell's neighbouring cells. It states that a live cell dies of loneliness if there are less than 2 live cells around it. Similarly, it dies of overcrowding if more than three neighbouring cells are alive. In other words, a cell will only "survive" by having exactly 2 or 3 neighbouring cells that are alive. For a dead cell to become alive, it needs to have exactly 3 live neighbouring cells, otherwise it stays dead. An example of the GoL automaton can be seen below, iterating rapidly through several states.

Another famous cellular automaton variant is a one-dimensional one, called the Elementary Cellular Automaton (ECA). This is the one we will be implementing in this post.

Each state of this automaton is stored as a one-dimensional boolean array, and while GOL requires two dimensions to visualise its state, this automaton requires only a single line of values. Because of this, we can use two dimensions (rather than an animation) to visualise the whole state history of this automaton. As with GOL, the state of a cell in this automaton is either 0 or 1, but while a cell in GOL is updated based on its 8 neighbours, ECA has its cell updated based on its left neighbour, its right neighbour and itself!

You can see examples of rules below, with the three top cells being the input of the rule, and the single bottom cell being the output, black being 1, white being 0. You also see the patterns each of them generates, with the initial condition being all 0's except a 1 in the middle cell.

You might wonder why the above rules have numbers attached to them. This is because each number between 0 and 255 directly corresponds to an ECA rule, and is thus used to name the rules. This subtle correspondence is shown below:

Any number in the interval from 0 through 255 can be represented in binary using only 8 digits (first arrow above). Furthermore, we can give each of these 8 digits an index based on their positioning (second arrow). These indices will naturally range between 0 and 7, which coincidentally are numbers that can be represented in binary using only 3 digits (third arrow). By interpreting these 3 digits as input, and the corresponding digit from our original number as output, we get the tertiary function we are looking for (fourth arrow).

Generating rules

Let's implement the above interpretation as a higher-order function get_rule that takes a number between 0 and 255 as its input and returns the ECA rule corresponding to that number.

We want it to work somewhat like this:

In the above example, running rule30(1,1,0) will combine the three binary values into a single number (110 = 6) and return the bit at that position (6) in the binary representation of 30. 30 is 00011110 in binary, so the function will return 0 (we do our counting from the right, and start counting from 0).

Knowing that the three binary input variables will be combined into one number, let's start by implementing such a combine function.

By left-shifting the arguments to their appropriate positions, then adding the three shifted numbers, we get the combination we were looking for.

The second important part of the get_rule function is to figure out what bit value is located at a certain position in a number. Let's therefore build the function get_bit(num, pos) that can return the bit value at a given position pos in a given number num. For instance, the number 141 is 10001101 in binary, so get_bit(2, 141) should return 1, while get_bit(5, 141) should return 0 .

get_bit(num,pos) can be implemented by first bitshifting our number pos positions to the right and then do a bitwise AND with the number 1.

Now it just a matter of putting these two functions together:

Cool! We now have a function that for each number between within our interval gives us a unique ECA rule that we can do whatever we want with. The next step is to visualise them in the browser.

Visualising rules

We will use a canvas element to visualise our automata in the browser. A canvas element can be created and added to the body of your html in the following way:

In order to interact with our canvas, we need a context. A context is what lets us draw shapes and lines, give things colour, and generally move around on our canvas. It is provided for us through the getContext method on our canvas.

The '2d' parameter refers to the context type we will be using in this example.

Next, we make a function that, given a context, an ECA rule and some info on the scale and number of our cells, draws the rule onto our canvas. The idea is to generate and draw the grid row by row; with the main part of the code looking something like this:

We start off with some initial collection of cells as our current row. This row, like in the examples above, usually contains all 0s except for a 1 in the middle cell, but it can also contain a completely random string on 1s and 0s. We draw this row of cells, then calculate the next row of values based on our current row, using our rule. Then we simply repeat drawing and calculating new steps until we feel that our grid is tall enough.

The above code snippet requires us to implement 3 functions: initial_row, draw_row and next_row.

initial_row is a simple function. Make an array of 0s and change the element in the middle of the array to a 1.

With our rule function readily available, the next_row function can be written as a oneliner. Each cell value in the new row is the product of our rule with the nearby cell values in the old row as input.

Do you notice our cheat in the line above? Each cell in our new row needs input from three other cells, but the two cells at each edge of the row only gets input from two. For instance, next_row[0] tries to get an input value from row[-1]. This works because javascript returns undefined when attempting to access values at indices that don't exist in an array, and it so happens that (undefined >> [any number]) (from our combine function) always returns 0. This means that we in reality treat every value outside our grid as a 0.

I know, it's not pretty, but we are making something really pretty on the screen very soon, so we are excused.

Next up is our draw_row function; the one that actually does the drawing!

This is where we are heavily depending on our context object, utilising no less than 5 different methods from it. Here is a quick look at each one, and how we use them.

fillStyle specifies what you want to fill your shapes with. It can be a colour, like "#f55", but also a gradient or a pattern. We use it to distinguish between 0-cells and 1-cells.

fillRect(x, y, w, h) draws a rectangle from point (x,y) with width w and height h, filled according to the fillStyle. Our rectangles are simple squares, but you might be surprised that they all are positioned in origo. This is because we use it in conjunction with translate.

translate(x, y) lets you move the whole coordinate system around. This persists, so it works as a great alternative to keeping track of the different positions of items. For instance, instead of calculating the position of each individual cell in our grid, we can just draw a cell, move to the right, draw a new cell and so on.

save() and restore() is used together with translate and other coordinate-transforming methods. We use them to save the current coordinate system at a certain point, so that we at a later point may return to it (using restore). In our case, we save our coordinate system before we start drawing a row and move to the right. Then, when we are done drawing the row and are all the way to the right, we restore, so we get back to our initial state. Finally we move down so that we are ready to start drawing the next row.

We now have all the parts needed for our draw_rule function. We will use this function on window.onload after having set up the canvas. We also define the parameters we need.

We extract the canvas dimensions as individual variables together with the number of cells horizontally. Then, we calculate the cell_scale and 'cells_down' so that the grid fills the whole canvas while keeping cells square. This way, we can now easily change the "resolution" of our grid while still keeping it within the bounds of the canvas.

...and that's it! The full code example is available on github and as a codepen:

See the Pen oNgZKqV by Kjetil Golid (@kgolid) on CodePen.

Onward from here

Having this setup now lets you explore all the 256 different rules one by one, either by iterating through each one by changing the code, or by letting the rule number be random at every page load. Either way, it is a great feeling to explore these unpredictable results within your own controlled environment.

Another thing to explore is letting the initial cell state of our automata be random rather than our static "only 0s, but a single 1" state. This will yield even more unpredictable results. Such a variant of our initial_row can be written like this:

You can see below how big of an effect this change in the initial row has on the output.

This is only one of the things you can change though! Why limit ourselves to two states? (Going from 2 to 3 states actually increases the number of rules from 256 to 7 625 597 484 987!) Why limit ourselves to squares? Why only 2 dimensions? Why only one rule at a time?

Below are some examples of ECA-based visualisations, but with an alternative draw_rule function, drawing lines in an isometric pattern rather than squares, then filling areas defined by those lines with colours. You can even choose to not display the separating lines at all, only showing the colours.

Taking it even further, one can start introducing symmetries, both rotational (middle row) and reflectional (bottom row).

If you find the above visuals interesting, feel free to check out this interactive playground, or even better, start from the code we've built here and try coming up with your very own cellular automata!

Good luck!

Cellular Automata in the Browser

The field of computer science is moving forward at an extraordinary pace. The phone in your pocket is becoming much more advanced thanks to these feats of engineering. The greatest challenge that everyone faces, however, is coming up with names in programming. Despite the massive accomplishments in AI and machine learning, the most difficult part of these tasks are, I believe, coming up with good names. BEM does not solve this issue, but it makes it easier by many degrees, and with the addition of a CSS preprocessor, it can make your CSS-code more readable and hierarchically sound than before.

So what is BEM? BEM is an acronym of the key elements of its main principles; Block, Element and Modifier. Let's explain each of these, and see how they can be utilized for a more consistent naming scheme for your CSS-classes.

Block, is a standalone entity, that in and of itself has an intuitive purpose. Examples of this could be, button, input, menu and header. These are standalone components that do not require any additional naming to be understood with regards to how they operate.

An Element is a part of a specific block. Meaning, without the _block, it starts to lose its meaning and its close attribution to the block is what makes it encompass a semantic tie to the block. An example of an element is item. item does not say anything meaningful about what you are giving CSS rules to. However, tied with a block, e.g. menu item, gives it a necessary context. Which in turn, provides guidance for the developer to its context, thus aiding in visualizing its implications.

The Modifier is self-explanatory (or autological if you want to be fancy). It described how something is modified. With regards to buttons think primary, danger and success. Similarly to Element, it does have a strong association with a Block, but could also be used to describe modifications done to an Element.

By combining these three principles, one could use this hierarchy to name things more logically and consistently. To combine these, an Element is prepended by two underscores __, and a modifier by two dashes --. Which leads to the syntax: Block__Element--Modifier.

Thus, when for example describing a menu, the wrapper is named menu and each menu item is named menu__item. If one of the menu-items is disabled, it will be named menu__item--disabled. By following these principles for giving class names, specificity remains low. Meaning, how the browser applies what CSS values to apply. By keeping it low, hopefully, your code behaves more like you think it should. Thus keeping "sledgehammers" like ID and !important out of the picture.

The examples consist of three Blocks: menu, page-content and button. menu, for example, has an element called item which could be in the modified state of disabled. By using the nesting capabilities of SCSS, the code gets a tree-like structure with regards to its logic, and making edits to specific modifiers and element while containing a base-block is how BEM really shines.

BEM does not entirely solve the problem of naming. However, it enhances the readability of code, keeps the specificity low and helps by making your CSS-class names more consistent. Thus, your life becomes a little easier and your code a little bit simpler to understand. Subsequently, your components are more flexible and easier to modify. Adopt BEM in your codebase, you will not regret it.

BEM - Naming made a little easier

Who is responsible for insights in a product team?

There are many pieces of information needed to navigate a product in its environment of users and competitors towards a goal. You could say that a product team is just like a group of pirates on a ship navigating towards their next treasure. They need to know about the nautical terrain, weather forecast and enemy ships to navigate successfully to their goal. Like in a product team, the team of pirates has their different roles and responsibilities. They each have their own separate piece of the truth about how the ship can reach the treasure. If the pirates don’t share their knowledge with each other they will never see the whole picture—and they risk stranding in the wrong bay.

On many pirate ships the responsibility of navigation falls on the cartographer (the designer)—the person who draws the map becomes responsible for navigational research as a whole. But in order to chart a good map, you regularly need to get out of the navigation room to get an overview of the area. More importantly, the surrounding area is just a part of the picture. The skipper (the tech lead) might know crucial information about the ship’s condition and the war pirate (marketing) might know best which routes that are safest to take. These insights are scattered on the ship and rarely shared between the pirates. When the ship is out at sea, the waves are rolling and the captain (the product manager) is drunk on rum, the team needs a way to regularly share and follow up necessary research to successfully navigate towards the treasure.

A team ritual

Over the last year we (a group of designers, strategists and developers at Bekk) have experimented and researched how to stitch treasure maps together for product teams that are out at sea. Inspired by other successful rituals like team retros we tried to make a ritual that would make product insights a strong part of the team culture.

Why a ritual?

We wanted to build an insight-based culture. To build a culture we need to change behavior and gather around a common belief. A successful way of doing this through history has been rituals.

We wanted to create an arena where different contributors could meet and share their insight with each other.

We tested different versions of rituals in our teams and workplaces (a variety of insurance, pension, over-the-top media services, social search service and transportation) over the past year and discovered that there are six important traits of a successful insight ritual:

Easy to implement – Better than something complex that you'll never use.

Actionable – No point of doing anything if it doesn’t bring results.

Boundaries – What is up for discussion now and what we’ll deal with later.

Goal – Keep the ritual aligned with the goal for the team or the product.

Progress – Avoid congestion by working with bite sized tasks.

Adjustable – No teams are equal so the ritual needs to have room for adaptation.

After several iterations, we arrived at the insight retro.

The Insight Retro

What: The insight retro is a small repeated meeting/ritual for teams.

Why: The purpose of the meeting is to create and motivate insight based culture.

When: Regular retros—from a few times a week to a few times a year. A regular retro should take 20-30 minutes.

Who: Everyone in the team who has a piece of the insight puzzle. If you’re in a big team, vary the combination of people from one retro to the next to avoid being too many.

How: Go through the following steps:

Status

Active experiments and initiatives. What are we currently investigating? Report back on initiatives from the last retro.

Discoveries

Findings and knowledge collected since last insight retro. What have we discovered recently?

Suggestions

Suggestions and ideas. What’s in sight? What should we investigate and which experiments should we do? Tip: set a limit of 1-2 ideas per person.

Priorities

Prioritize the most important experiments and initiatives. Dot voting or counting hands – keep this simple and goal-oriented.

Ownership

Each problem or suggestion should have an owner. This person is responsible for reporting back the results in the next retro. That doesn’t necessarily mean that the owner is responsible for doing the actual research.

Tip 1: To make it easier to prioritize you can try to make tasks (experiments and initiatives) in the same size. Try to make the task big enough to be of true value and small enough not to depend on everything else.

Tip 2: If you are trying this for the first time, we recommend trying with 5 people or less. Then make up your mind who should be involved.

Try it on your own ship!

If you feel your team has an incomplete treasure map that needs to be stitched together, you should try an insight retro! We can’t imagine an easier way to engage the whole team around collecting and managing product insights. Although you and your fellow pirates might have just returned from this year’s biggest sea battle (julebord), January seems like a good time to try out a few insight retros before your team set sail towards your next treasure.

If you try this with your team and develop it further, we would love to get some feedback 👆🔝

Your product team needs an insight retro

On one of the darkest Sundays of the year, we again take a step back, and give you another list of interesting people we follow. Today we pay respect to a few people that deserve to be listened to. Of course there are others, but these stand out.

@troyhunt

Troy Hunt is probably most known for being the author of the Have I Been Pwned? project, where people can check if their email credentials have been leaked in various data breaches. By collecting the database dumps and organizing these in services where users can check their own information, such as username or password. Recently he has announced that we will sell off the service. At time of writing, no suitable buyer has been found.

In addition to this Hunt is an active author and speaker. He will be attending NDC Security next year, be sure to catch him there. https://www.troyhunt.com

@briankrebs

On the KrebsOnSecurity you can find the work of American author and investigative reporter Brian Krebs. Krebs started his security reporting on The Washington Post in the late 1990-ies. Later he became known for his investigating reporting on crybercriminals. His high profile has made him a target of attacks on the internet, like the ditributed denial of service attack in 2016. He still post dayily on his blog. https://krebsonsecurity.com

@SGgrc

Steve Gibson got his bragging rights designing light pen hardware for Apple II, creating software for recovering ZIP-drives and hard drives. Today he is probably mostly recognized for being co-host of the Security Now podcast, where he reports on the latest news in addition to a deep dive into a security related topic. He is also the brain child behind SQRL, an alternative to username and password, which we wrote about in https://security.christmas/2019/2

22-Dec

Today's posts