Hvordan jobber du brukerorientert og med brukerbehov når brukerene dine er langt unna? Langt unna både verdikjede distanse og langt unna i tid? Finnes det noe enkel løsning? Linda Halvorsen, Jens Andreas Huseby og Moquan Chen diskuterer og spekulerer i denne kaffepraten.

Hvordan jobbe brukerorientert når du ikke har brukere?

Or –– how to upgrade your code base from JDK 8 to 11 and beyond.

We've previously covered what's new in JDK 11 and 12, 13 and 14 (part I, part II) and we're hoping you've found some features you want to start using. However, it is not uncommon for huge, critical Java applications to still use JDK 8. It's definitely time to leave JDK 8 behind. I mean, it is over 5 years old, and you're missing out on a lot of good stuff.

While upgrading should be fairly straight forward, if you are using certain parts of the JDK in your code base you'll actually encounter some compile errors. This is because a few packages were removed from Java SE in JDK 11 after being deprecated in JDK 9. Fear not, though, because most of the packages have released as Maven artifacts.

The removed packages are:

JAF (java.activation)

CORBA (java.corba)

JTA (javax.transaction)

JAXB (java.xml.bind)

JAX-WS (java.xml.ws)

Common Annotations (java.xml.ws.annotation)

Replacement Maven artifacts:

JAF (java.activation)

CORBA (java.corba)

In the JEP it is stated that «There will not be a standalone version of CORBA unless third parties take over maintenance of the CORBA APIs, ORB implementation, CosNaming provider, etc.». However, you may have a look at Glassfish CORBA ORB. According to the website it «[…] complies with the CORBA 2.3.1 specification, and with the CORBA 3.0 specifications for the Interoperable Name Service and Portable Interceptors. It includes both IDL and RMI-IIOP support».

JTA (javax.transaction)

JAXB (java.xml.bind)

JAX-WS (java.xml.ws)

Common Annotations (java.xml.ws.annotation)

💡 The version numbers above are the newest versions as of December 21st 2019. Make sure to verify you're using the newest version, and keep them them updated, by using e.g. maven-dependency-plugin.

So long, Java 8

Santa needs your help. The usually ever so helpful elves have decided to play a trick on him, and have made all the toys run around, resisting getting wrapped in the lovely Christmas wrapping paper he just spent all day picking out. Can you help Santa catch and wrap all of them?

See the Pen <a href='https://codepen.io/mfeiring/pen/eYmJNZy'>CSS Christmas game</a> by Mira Feiring (<a href='https://codepen.io/mfeiring'>@mfeiring</a>) on <a href='https://codepen.io'>CodePen</a>.

Did you catch all the toys? Well, perhaps it wasn't really the most challenging game. But might I interest you in a short rundown of how the game is build? Read on my dear friend. But beware: This might not look its best on a small devices.

The setup

To start I've created a pretty basic layout. In contains five checkboxes, with corresponding labels, linked together with the for and id attributes. I've increased the size of the input fields with the zoom property (this doesn't work in Firefox unfortunately). By scrolling down a bit, you should see the success screen.

The counter

First up is the counter for how many gifts have been wrapped, starting by removing the static numbers from the HTML. Then moving on to the CSS "magic" – the counter property.

The following snippet will increment the counter gifts for each input field in the DOM, as well as the counter gifts-wrapped for each of them that is checked.

In order to display these counters, we need to use a pseudo-element and the counter() function, which returns the current value of a counter as a string:

The counter should now increment for each checkbox that is, well, checked.

The gifts and the wrapping

A game about checking boxes isn't really that enticing, so some styling of them are in order. It isn't really possible to do anything with the standard styling of checkboxes though. So we'll have to do some magic with the label elements instead, using an attribute selector to differentiate between them.

"But couldn't you just add those emojis to the HTML?" you might ask. And the answer is "Yes, yes I could. But then I wouldn't be able to do this!"

And voilà! Now checked boxes equal wrapped gifts 🎁

Finally, let's prevent the possible unintended "unwrapping" of presents, as well as (visually) remove the checkboxes:

‍

The movement

So far the game is ridiculously easy. So we'll make the toys move around a bit using a CSS animation.

Here, we have an infinite animation, in which the animation-direction is alternate, taking the toys 5 seconds to reach the bottom, and the another 5 seconds to go back up again. The calc function is used to prevent the toys from moving beneath the counter, by calculating the height of the viewport (100vh) minus the height of the counter container (3rem).

By adding a few more keyframes to the animation, we can make the movement a bit more erratic:

‍

And being a bit creative with nth-of-type selectors, we can give the toys tome individuality in their movement.

Let's also make them stand still when they're caught and wrapped:

‍

The winning

In CSS, the general sibling combinator (~) allows us to select an element that comes after a given other element (though not necessarily immediately after), that is within the same parent element. In order to show the "Good job!" screen upon completing the game, we need to select element containing this screen, iff it is preceded by five checked input fields, like so:

And to prevent anyone from just scrolling down to this screen, let's prevent scrolling completely:

‍

Wrapping up

And that's it! We've created a game using only CSS and HTML.

For more inspiration, I'd highly recommend checking out the collection of CSS games that Una Kravets has compiled on Codepen, as well as the talk she held on CSSconf EU in 2017, both linked down below.

Wrap it up! – Creating a CSS game

Maybe you’ve heard of Fetch? The promise-based Web API for making network requests in the browser, and a favourite amongst developers. If you paste fetch(‘https://www.google.com’) into the Chrome DevTools console and run it you can take a closer look at how it works. Sadly, you can’t do the same in Internet Explorer. It won’t work. That is because Internet Explorer has not implemented support for Fetch. Or Promise for that matter.

As developers of things on the web we want, and are often required, to support all major browsers out there. More users often converts to more success. So, of course we want to invite everyone in. The problem arises when our users, many unknowingly I suppose, are using old or specific browsers that do not support the latest and greatest features - which we, as developers, are longing to use. Polyfills to the rescue!

What is a polyfill?

If the browser that is running your code do not support the features that you want to apply in your modern JavaScript, like Object.assign() or fetch(), a common practice is to provide the pieces of code that is missing yourself. These pieces of code are called polyfills. It can be as easy as finding the required implementation yourself and adding it to your source code.

Here is an example of MDNs suggested polyfill for Number.isInteger():

Just add this to your Javascript, and make sure its declared and on a reachable scope before it is used, and you’re good to go.

There are also many polyfills that can be found on NPM, like the popular polyfill for Fetch which removes the burden of you having to find the perfect implementation yourself.

But how to know if a feature is supported or not? I would recommend using the MDN Web Docs from Mozilla Developer Network. In addition to showing browser compatibility, it also contains great documentation and often suggested polyfills. CanIUse is another great alternative, offering a quick and easy way for looking up compatibility for a lot of different features in both Javascript, CSS and more.

If your code base requires a lot of polyfilling, maybe also some transpiling, adding tools like Babel.js to your build setup enables an easy way of securing that required polyfills are available when your code is running.

Polyfilling is often confused with transpiling. These two techniques solve different problems. While polyfilling adds a piece of missing code, transpiling transforms your code and enables you to write syntax that may be unsupported in certain browsers, like arrow functions and class declarations.

When can we stop worrying about polyfills?

As long as there are multiple major browsers from different companies doing things their own way, there is no way to guarantee that they are up to speed on the latest standards like ECMAScript and WHATWG.

Nonetheless, the need for polyfilling recent years have mostly been related to unsupported features in Internet Explorer. The market share of Internet Explorer is dropping slowly but steady, but is said to be supported throughout the lifespan of Windows 10. The good news is that Microsoft plans to release the brand new Edge Chromium, which is based on Google Chrome's open sourced engine, 15th of January next year. Let's hope Windows users adopt this version of Edge faster than the last, so that the era of error messages like X doesn't support property or method 'y' might come to an end.

A short recap on polyfills

Becoming “data-driven” has been a stated objective for many organizations over the past few years. However, according to Harvard Business review most companies are failing to achieve this.

There are a variety of possible explanations for the failures to become a data-driven organization. A lack of available resources with the right skill set, an outdated data platform or lack of management buy-in, are some of the reasons we have witnessed.

We have also seen how organizations have tried different approaches to reaching their objectives. Some successful and others not. Spending three years on building a pre-defined data platform and hiring 50+ data scientists falls into the latter category.

What’s typical for those who succeed is that they work systematically along four dimensions. Typically, all organizations are placed on a scale from non-existing to mature within these four dimensions, which are described below.

###Infrastructure

The requirements for infrastructure have changed as a result of the need for increased volume, speed and data variation. Different organization’s maturity ranges from operational databases with ad hoc extraction of data, to gathering and consuming a variety of data sources in real-time.

###Competence

Converting data into actions requires the right analytics competence. At the lower end of the maturity scale, business developers only interpret reports created by a few employed BI analysts. At the other end you have analytical and self-propelled business developers, along with highly skilled data scientists who work exploratory.

###Governance and organization

Organizations who manage to take full advantage of available data are usually organized in product teams. Data scientists are then distributed across these teams, where they spend most of their time, while maintaining and updating their knowledge through a professional community.

###Process and culture

Less mature organizations mostly use data in static reporting and base many decisions on intuition and experience. Data-driven-organizations however, work hypothesis driven and dedicate time to exploratory work, and they will require data to support every leadership and board decision.

Becoming a data-driven organization requires work along all these dimensions, and there is not one single answer to what is the right approach for you. Analyse where you are, set tangible and realistic ambitions within the four different dimensions and start getting your hands dirty with a few business-relevant use-cases. Some initiatives will succeed, others not. Scale those that succeed, and suddenly you are data-driven before you even realise it.

Become a data-driven organization

Folka i Monotype, kanskje verdens fremste fonteksperter, publiserte nylig en trendrapport om typografi. De mener at bruken av variable fonter kommer til å øke dramatisk de neste årene. Men hva er egentlig variable fonter? Og hvorfor har det plutselig blitt så aktuelt akkurat nå?



Variable fonter, sa du?

Variable fonter er en ny måte å anvende typografi i digitale flater, noe som gir en enorm fleksibilitet og muligheter for lekenhet i hvordan vi viser frem og setter tekst i grensesnittet. Og det er jo egentlig et paradoks at vi i mange år har designet dynamiske grensesnitt, mens typografien og det tekstlige innholdet i stor grad har vært begrenset av tradisjonelle "statiske" fonter. Med variable fonter kan også teksten bli dynamisk.

I motsetning til tradisjonelle fonter, der hvert snitt av en skrifttype lagres som individuelle fontfiler, vil de i en variabel font samles i én enkelt fontfil. Dette gjør at du får færre filer å holde styr på, færre kilobytes, og mye, mye mer typografisk frihet!

Filstørrelsen blir mindre fordi hvert tegn tegnes som et omriss. Punktene som tegner omrisset har instruksjoner for hvordan de skal oppføre seg for å f.eks lage en annen vekt. Deretter blir de individuelle stilene interpolert, som betyr at de blir tegnet dynamisk i browseren. Interpolering kan foregå i veldig små steg langs forskjellige akser, som f.eks en vekt-akse. Dette gjør at du kan finjustere innstillingene dine til akkurat det du trenger. Hvis du leter etter en vekt et sted mellom normal og fet, eller du vil at kursiv skal lene seg litt mindre, så er dette fullt mulig. Og det er faktisk nesten ingen begrensning for hvilke akser skriftdesigneren kan legge til i fontene. Sjekk ut FS Kitty fra Fontsmith, her kan du justere akser for både skygge og omriss.

Hvorfor er variable fonter 🔥 akkurat nå?

Vel, teknologien er ganske ny, faktisk bare tre år gammel. Variable fonter er en evolusjon av OpenType, formatet vi har brukt i flere år. Det er ikke før det siste året at flere av de største browserne har lagt inn støtte for formatet, og for ikke lenge siden gjorde både Figma og Sketch det mulig å designe med variable fonter. Så nå er tiden inne for å eksperimentere!

Her er en liten gavepakke med noen av mine favorittlenker om variable fonter. God jul 🎅🏽

1. 5 type trends for brands to consider in 2020\ I Monotype sin trendrapport, lister de ut 5 typografitrender for 2020 og årene som kommer, en av trendene er selvfølgelig variable fonter.

2. Smashing Podcast Episode 5 With Jason Pamental: What Are Variable Fonts?\ Jason Pamental er kanskje den mest tydeligste talspersonen for digital typografi, så han har naturligvis snakket om variable fonter både her og der, senest i Smashing magazine sin podcast.

3. Phase is a generative type concept\ Test ut hvordan variabel fontteknologi kan manipuleres i sanntid eller med lyd via stemmesignal.

4. The Font Gautlet\ Det sveitsiske designstudioet Dinamo har laget et verktøy hvor du kan teste variable fonter rett i browseren. Her kan du velge blant eksisterende, eller laste opp egne variable fonter og få en animert forhåndsvisning av alle aksene de inneholder.

5. V-fonts.com\ Er det et sted du skal gå for å få tilgang til, og teste ut variable fonter etterhvert som de blir lansert så er det på v-fonts.com.

6. Movement\ Movement er en variabel font som er inspirert av dansebevegelser. Ganske stilig!

Slipp fontene fri!

As web developers, we are often validating if the data coming from the frontend is valid per our domain logic.

To create a smooth experience for users, we want to respond to a request with either a success, or a failure, with a list of all the validation rules that didn't pass. This way the user does not have to post the request multiple times, getting a new error each time.

We want our validation functions to take any input, and give out either a valid result type, or an error describing which of our constraints was broken.

By modelling the result from our validation function in a type that can represent both the valid and invalid data, we say the function is total. This will let us reason about the validation rule by just reading the signature of the function, i.e. a function from a String to either a Username, or an error message.

Define a domain

At my project, we are using Scala and Elm in our web stack. While frontend validation is nice, we must use backend validation to enforce our domain rules. Let's say we want to create a user registration page, where a user has a name, age and a phone number:

Validation functions

Then we create the Validated type to represent either the valid instance of type A, or a list of error messages that we can display on our registration page. We create a type alias for the Either type which is generic regarding the success type, but locks the error type to a list of strings:

We could also represent errors as their own type, but let's stick to strings for this example.

Then let us write our validation functions. The Either type is a sum type that can either be a Right for the success case, or a Left to hold the error case. Here I have first created a generic helper function cond that just wraps the input in Right if a predicate pass, else the error is wrapped in a Left:

We then create some simple validation functions for all the types needed to create a valid User. Let's say names cannot be longer than 3 characters, the minimum age is 18, and phone numbers must be exactly 8 characters long:

Validation using Monad

Because Either is a Monad in Scala we can implement our validation function using the map and flatMap properties provided by the Monad type class:

Using for-comprehensions, which are just syntax sugar for flatMap and map we get very nice syntax for defining a sequence of monadic operations:

The problem with using monadic properties for validation is that they model sequential computation, and if one of the validation functions fails, the resulting Left value will be passed on, and subsequent calls to map or flatMap will do nothing but pass along the Left value. The result is that even if all validation rules was broken, we only get one error message.

Validation using Applicative Functor

We need some way of composing the validation functions in parallel. Let us introduce another type class, the Applicative Functor. The Functor type class gives us the ability to apply functions to a value inside some context using map. The Applicative Functor type class extends on this ability with two properties. pure is the ability to create a context with the given value. For the Either type we are using as our context, this would simply be constructing a Right value:

The other property required by the Applicative Functor is the ability to provide some way to combine two the values inside multiple context together. map2 lets us supply a function that can combine the types inside the contexts, and the related function, product lets us put both values from two contexts into the same context.

By implementing this type class interface for our context, in our case the Either type, we can derive other useful functions, like map, and product. map is simply a map2 where we don't care about the second parameter, and product is just a map2 supplied with the identity function:

Let us give the Either type these properties. Scala uses implicit parameters to implement type classes, and the syntax for defining a type class for the Either type would look like this:

There is a bit of syntax to decode here, and some of it is because of how Scala uses implicit parameters to implement type classes. The important parts are on the second line, where we create an Applicative instance for the type Either of the error type E and any value type.

We can see from the implementation of map2 that the result from our validation functions will both be evaluated in an exhaustive pattern match. In the first case of the match statement we have our happy case, where we can apply our function.

The second case, where both values have failed is the most interesting one. We want to somehow concatenate the errors from both, and because we want to use a list of strings, we could have implemented this type class for our Validated type, and then we could simply use the built-in functions on lists to add them together.

However, because we have already written about Monoid, we know that this type class gives us the ability to combine two objects of the same type. By constraining the error type E in to Monoid, we can later change the error type in our Validated type without changing any code!

We can now implement our validation function. We use product to combine all our results into the same Validated context, and then map the nested tuples of the success case to a validated instance of User:

We can here see the use of the implicit parameter. The compiler searches for an instance of Applicative the type Validated, which we remember is the same as Either[List[String], A]. If the compiler finds the Applicative instance for Validated, and the Monoid instance for List, the code will compile.

We can now run our validation, and it will create a list all the validation rules that failed.

However, the code looks a bit messy. Luckily this pattern can be generalized and added to the applicative type class as map3.

Then we are all done?

This does look a lot cleaner, but there is still one problem, what if we want to add another field to the User type? We would then need to implement map4, and so on.

In our real-world projects, we use a library for Scala called Cats, which comes with all batteries included. This library provides a large collection of type classes, like Monoid, Functor, Applicative and Monad, and instances for the built-in types in Scala.

To implement my validation function using Cats, I would only need to import the implicit type class instances from Cats, write the three validation functions, and apply them using parMapN.

I hope I could convince you how using patterns from functional programming can help us solve some common problems in a terse and readable way.

Functional Data Validation

GitHub actions is the new kid on the block. Lets look at a first try in implementing build and release-functionality.

At Digipost we use several public open source projects that we manage and release on a quite regular basis. We need to simplify the release process for security, release ease and easier onboarding of developers.

Lets assume you have all you need to deploy to Maven central. In addition you need a configured nexus-staging-maven-plugin or you can use this parent pom so that the artifact is released without you needing to log in to oss.sonatype.com and click close and release.

Leveraging open source for our solution

I will not give a deep explanation of what GitHub actions is. But what is nice to know is that it can run docker images on linux. With that we can setup an environment that can execute maven. We forked a nice action already in progress and modified it into the action we need. GitHub, at this time of writing, does neither have organisation secrets or the ability to add repository secrets with api. We wanted to save time and have only one secret per repository in stead of 4. In addition we wanted to deploy both a SNAPSHOT and a release with the same action. Build version is set automatically from either tag-name of branch-name.

Now, we need to call this from a workflow added to the repoitory. We made two workflows pr project: build.yml and `release.yml

Usage

For every push to all branches, we build the project on both Java 1.8 and 11. In addition we build a version on Java 11 and deploy this to Sonatype snapshot-repository. For every tag (GitHub-release) we will build a release and this is published directly to maven central.

PS: Sick of all the download progress in Travis, GitHub actions or what not? Starting with Maven 3.6.1 --no-transfer-progress will remove the noise leaving only nice readable output.

OSS Java and Github actions

In business travel, a road warrior is a person that uses mobile devices such as tablet, laptop, smartphone and internet connectivity while traveling to conduct business. The term spawns from the movie Mad Max 2, starring Mel Gibson.

This is in no way a definitive guide. Before you strap on your tinfoil-hat, rember that a few of these tricks perhaps are a little overboard. We try to provide you a guide for moderate to high security against the everyday band of marauders.

Use a VPN

VPN is a safe and easy way to protect yourself. Often your company will provide one, and if not, you can check out commercial alternatives. Stay clear of "free" alternatives, it is advisable to also check how long they preserve logs.\ The point of a VPN is to generate a private network between you and a machine you trust, where your traffic is “tunneled” through this network.

Use your cellphone as a roaming hotspot

Instead of connecting to Free public Wi-Fi, which we know that in most cases are run by the bad guys and provides no security, you can use your cellphone as a roaming hotspot for your devices. Watch out for high data fees.

Disable biometric unlocking of device

If you know you are going to enter a country that allows the authorities to force you to give up your biometric credentials, like fingerprint or facial id, this should be disabled. You are still allowed to "forget" a PIN or password.

On a side note; some countries may not allow you to enter a device into the country which you cannot unlock, so this can result in you being denied access to the country.

The Mikado trick

If you at some point in your travels have to leave your laptop, let’s say for example in the hotel room. Take a bunch of Mikado sticks, drop them over your laptop, and snap a picture.

The randomness generated by the sticks are impossible to replicate and will provide an easy enough verification method to check if something has changed from when you left it.

When leaving your laptop, you should propably turn it off. Rob Fuller showed that with a 50$ device, anyone can extract creds from a locked laptop.

Password manager travel mode

If you are using a password manager (and you should!!), you can enable travel mode. This removes password vaults from your devices, except those that you mark as “safe for travel”. So, if you have access to a company password vault, you should perhaps remove that vault before entering a country that would like that information. \ Last year Hans Kristian wrote about password managers, and why you should use one.

Buy it in the store

We know from the files Edward Snowden leaked that NSA has the capability to intercept a parcel from i.e. Amazon to do hardware modifications. This was shown with certain Cisco routers.

If you go to your local computer store, and buy the device there, and make sure that it does not leave your sight from when you pick it up, you can trust that it has not been tampered with.

Visiting places that are known as a bad place?

If you are visiting places or countries that puts you at high risk of information theft or hacking, you should use a dedicated phone/tablet/pc for travels that gets wiped after your visit.

Wait to turn on your devices until you have left the airport, as this is where intelligence services have the highest presence.

Use a USB-condom if you need to charge your phone. A public charger is never a good thing..

Final thoughts

Keep your devices updated with the latest security updates. This is important for OS, but also with updates for your apps.

Encrypt your devices, and do regular backup to a secure off-site location.

Safe travels for the road warrior

Did you know that you can change the scope of lambdas? It’s pretty awesome. In Kotlin you can change the scope of lambdas by using a concept called receivers, and the Kotlin standard library does it all the time.

Let's take a closer look at the familiar apply function.

With apply you don't have to reference the object you are working with, and instead, you can use properties such as username directly. Cool! So how can we implement our own apply?

Lambdas with receiver types is the answer!

This post assumes basic knowledge about lambdas. Head over to On wavelength with lambdas for a short introduction.

Ok, so let's take a closer look at whats happening in the example above. The caps function takes in a user and a lambda with some unusual syntax: text: User.() -> String. What's happening here? We have a lambda function type prefixed with a type! The prefix is a receiver type, and in our example that is a User.

But what does this mean? What is a receiver? Receivers assign a scope, or a receiver type, to a lambda. This means that we can change what this is inside a lambda. Having a User as a receiver gives us the power to call everything on a user object without having to explicitly reference it.

The second thing that might look slightly weird at first glimpse is the way the text lambda is invoked. It's called directly on the user object as if it were an extension function. When working with lambdas with receivers we have some nice syntactical sugar; since the parameter user has the same type as the receiver, we can call the lambda directly on it. This means that in the context of our lambda user.text() is the same thing as text(user).

Our caps function doesn't quite look like apply yet, but we are getting there. On the other hand caps looks very similar to the with function in Kotlin, which is not a keyword, but is in fact a function in the standard library. Let's compare them.

Both caps and with changes the context of the code block allowing us to refer directly to properties. But our goal was to implement apply, not with.

Extension functions to the rescue! Let's change our caps function to be an extension functions and see what happens.

In extension functions this refers to the object it's invoked on and since we are extending User, this is actually a reference to a user. When using receivers types this allows us to call text() directly without supplying the user to it.

Now we're talking. With that change our caps function look and behave much more like apply. Using extension functions and lambdas with receiver types we can make a lot of cool code snippets, and even create amazing domain specific languages!

Receivers make it possible to create some pretty interesting things, but remember that with great syntactical powers comes great responsibilities.

Receivers

Teslas are the most technologically advanced cars on the market right now. One interesting feature is called Sentry Mode, it's a built-in security system that uses 4 (out of 8) cameras to continuously monitoring the surroundings of the car. Unfortunately, there is no way to watch the videos remotely, they have to be stored on a USB drive that’s plugged into the car. That’s very impractical, it takes only a few days before there are too many videos and you end up just ignoring them.

In this post, I want to describe how I’m trying to use video content analysis to determine if there is a person on any video and then send a notification to the owner. I’m using Azure for archiving the videos and AWS to process them.



Archiving dashcam footage to the cloud

I use a Raspberry Pi Zero to emulate a USB mass storage so that the car writes dashcam footage to it. There is an open source project called Teslausb that can be used for that. I recommend this fork because it has a prebuilt image and a nice one-step setup. I’m using Azure File Storage because it supports SMB protocol out of the box. Here is my configuration:

Once the Raspberry Pi is configured and plugged in, it starts archiving Tesla Sentry dashcam footage to Azure File Storage as soon as the configured Wifi network is accessible. The next step is to run video content analysis to determine if there is a person on any of the videos.

Video Content Analysis

Azure Computer Vision would be a good fit here since I’m already using Azure File Storage, but unsupervised object detection only works on images and not videos 😔. That would require preprocessing all videos with ffmpeg for frame extraction and then running Azure Computer Vision on thousands of frames. A show-stopper for me.

Amazon Rekognition is able to perform unsupervised object detection on videos stored in a S3 bucket, but it’s a little bit more complicated because it's asynchronous. Video analysis is triggered by calling the StartLabelDetection operation and the completion status of the request is published to an Amazon SNS topic. The result is retrieved by calling GetLabelDetection.

Putting it all together

I use Azure Container Instances (ACI) to copy videos from Azure File Storage to S3 and then trigger AWS Rekognition. Then a Lambda function is triggered and the result for the operation is retrieved. If a person has been detected, I use pushover.net to send a notification to my phone.

Results

It turns out that detecting people on videos is not as easy as I thought and quite unreliable. There are a few things I want to change in my current solution:

Azure File Storage doesn’t provide triggers/events, so my container has to run every 5 minutes to check for new files. I’m going to use RClone to archive the videos directly to a S3 bucket and use a Lambda function instead of Azure ACI, thus running everything on AWS.

I only use the front camera for the moment, I’m going to merge all clips together on the Raspberry Pi before archiving to the cloud. This python project seems to do exactly what I want.

It should be possible to extract all the frames containing a person and then run face recognition on them.

It should be possible to view the videos remotely, right now I need to mount the Azure File Share on my computer in order to access the video archive.

Tesla Sentry with AWS Rekognition

I started working with React and TypeScript about six months ago. Since then I’ve learned a lot, and there are a few things I wish I knew when I started out that would have saved me from messy solutions and some frustrating moments. Here are the ones I think are the most useful.

Type inference

You don’t have to explicitly type everything, TypeScript will use the type of the initial value of a variable. It works for more complex types too.

If you realise you need the type in retrospect, you can can go back and declare the type, or you can use typeof to declare the type based on the inference.

Union types

When I started out with typescript and React I was mostly familiar with typed languages through Java so this was the first time I came across union types. If you haven’t come across it before it basically allows you to say that a variable can be one of several types. It’s useful if you want to avoid type inference but still want a variable to be able to have more than one type. For instance if you want to initialize a state with a null value

Discriminated Unions

In the same vein as the tip above, here’s another tip for typing which incorporates union types. If you make a union type of several types that have one common literal type that you can use to tell the types apart you have what’s called a discriminated union type. Useful if you want your components accept different sets of props.

So that’s a few features I really wish I was familiar with when I started out using react and Typescript. If you want to read more I highly recommend these cheatsheets and the official typescript docs

Happy holidays :-)

21-Dec

Today's posts