--- title: "Knitting + Machine learning = ❤️" url: "https://www.bekk.christmas/post/2019/22/knitting-%2B-machine-learning" authors: ["Henrik Aasen Kjeldsberg"] published: "2019-12-22" updated: "2024-11-07T13:05:28Z" language: "en-US" type: "article" tags: ["Maskinlæring"] --- # Knitting + Machine learning = ❤️ When knitting met machine learning, an untold love story. ## In short Machine learning can translate a picture of a knitted pattern into the instructions needed to reproduce it on automated knitting machines. The approach defines a knitting language, uses GANs to generate synthetic training data, and trains a CNN-based image-to-recipe model called Img2Prog. - **For:** Machine learning enthusiasts and knitting hobbyists - **Good to know beforehand:** Basic familiarity with machine learning concepts _AI-generated_ You heard it here first: machine learning and knitting are officially in a relationship! Yes, knitting, the act of transforming yarn into textile and fabric. Thanks to a group of researchers at the Massachusetts Institute of Technology, there now exists software which when fed a picture of a garnet, produces instructions for that exact pattern, used by automated knitting machines – they call it _Inverse knitting_. While on the subject, they are also the brains behind the project _Knitting skeletons,_ a software which lets you design your own knitwear, without any knitting or coding experience required. Amazing! Let's take a look at _Inverse knitting!_ Today's standard is using advanced knitting machines for mass production of garnets and textiles. However, these require pre designed recipes, and are maintained by low-level programming code, hence requiring a certain expertise to develop and maintain. This was the motivation behind the development of _Neural Inverse Knitting_; the intersection between machine learning and knitting. So how does one combine machine learning and knitting? You might consider this an unlikely combination, however, there is a logical approach behind the problem. In essence, this is a problem within pattern recognition - a common problem in the world of machine learning. The solution presented in the article employs methods such as _generative adversarial networks_ (GAN), and _convolutional neural networks_ (CNN) to automate the generation of knitting patterns for industrial use. In particular, GAN was applied to generate artificial textile samples used for training the network, whereas CNN is used to generate the main _image-to-recipe_-model, named _Img2Prog_. We will proceed by looking at the steps to solving this unconventional combination of fields. ![Knitting language](https://cdn.sanity.io/images/ah2n1vfr/bekk-blogg-prod/b0f39f400f3a6006dd86f50ffe0cd666b341da0a-512x50.png) - First, a knitting language is defined, describing knitting patterns. This knitting language consists of 17 operations, and is possibly recognisable by knitting enthusiasts. Here we find operations such as **k**nit, **p**url, **t**uck, and **m**iss to name a few. The rest of the operations are listed in the figure above. - Second, we need training data. For this particular case, the training data consisted of several pictures of patterned clothing. However, after collecting over 2,000 real photographs of patterns from clothing and textiles, the authors realised they had to resort to artificial data. In combination with real data, artificial data plays a strong role in data science and machine learning, and is both quick and inexpensive to produce. To overcome the current challenge, the real data was melded with the synthetic images, to create regularised synthetic images of knitting patterns, using GAN. This way there was a consistency between the real and artificial data. - Finally, the model, a prediction network, was trained using CNN. For reference, the entire process is visualised in the figure below. The resulting predictor, named _Img2Prog_, could now be fed an image of a pattern as input, producing the recipe for the corresponding pattern as output. Amazing! ![](https://i.ibb.co/wgxqHzD/knit1.jpg) If you still have some Christmas gifts left to buy/make, you now know what to do. Find your favourite theme, take a picture, apply _Img2Prog_ and start knitting! Merry Knitmas :D ## Relevant resources recommended by the author - [Article](https://arxiv.org/pdf/1902.02752.pdf) - [Homepage of Deep Knitting](http://deepknitting.csail.mit.edu/) - [Homepage of Knitting Skeletons](http://knitskel.csail.mit.edu/)