About Adversarial.io

Adversarial.io is an easy-to-use webapp for alter­ing image mate­r­i­al, in order to make it machine-unreadable. 

Through intro­duc­ing per­tur­bance, adversarial.io seeks to ques­tion and sub­vert auto­mat­ed image recognition.

For each uploaded image an Arti­fi­cial Intel­li­gence »neur­al net­work« cal­cu­lates the descrip­tion (i.e. »tab­by«).

Then an ader­sar­i­al algo­rithm cal­cu­lates a noise pat­tern, that moves the descrip­tion class towards the next class (i.e. »lynx«).

This adver­sar­i­al noise is a slight alter­ation, mov­ing the machine per­cep­tion, over a cer­tain thresh­old towards anoth­er descrip­tion of the image. 

While machine vision is tricked, the human eye is able to com­pen­sate for the intro­duced noise.

Read more on the blog.

From Tactical Media to Strategic Infrastructures

Adversarial.io posi­tions itself in the tra­di­tion of tac­ti­cal media inter­ven­tions by 1990s.

»Tac­ti­cal Media, the post-Berlin Wall child of mul­ti-media and inter­net prac­ticed by activists, design­ers and artists, hack­ers, and video enthu­sisas­ts, refused to make his­to­ry« (Lovink/Rossiter 2018:18)

Yet, instead of a hit-and-run media action, adversarial.io seeks to build infra­struc­ture strate­gi­cal­ly, look­ing for pos­si­ble alliances against the evil of image sur­veil­lance. If this res­onates with you, join the forces!

Adversarial.io looks for civic or pub­lic, non-com­mer­cial pro­cess­ing infra­struc­ture for machine learn­ing tech­niques. Let us know.

The project is ask­ing ques­tions about the mate­ri­al­i­ty of data by expos­ing how com­put­er vision works and how it can be tricked. It is sub­vert­ing abstract tech­no­log­i­cal process­es by mak­ing them vis­i­ble and explain­ing them. And it is chal­leng­ing nor­ma­tive assump­tions, by call­ing out the norms – in this case the image classes.

»If apophe­nia is the human ten­den­cy to per­ceive mean­ing­ful pat­terns in ran­dom data, under­fit­ting is a sort of machine apophe­nia. Machine apophe­nia hap­pens if a sta­tis­ti­cal mod­el sees a pat­tern that is not there, that is, if it reads noise as sim­i­lar to an exist­ing pat­tern.« (Pasquinel­li 2019:11)

Project goals

1.) Allows you to test your own images against the Incep­tion V3 pat­tern recog­ni­tion model.

2.) Cre­ate a scal­able, easy-to-use solu­tion, which demon­strates how AI pat­tern recog­ni­tion fails and how stealth meth­ods can be deployed.

3.) Edu­cate.

FAQ – Frequently Asked Questions

Don’t you think, you support automated computer vision by supplying it with test cases?

Cur­rent­ly no. At the moment we’re build­ing on pub­lished research that has been under­tak­en by tech com­pa­nies in order to strength­en their prod­ucts against adver­sar­i­al attacks. 

We are not yet invent­ing new ones and just make exist­ing adver­sar­i­al attacks avail­ble to a broad­er public.

The largest test case for adver­sar­i­al images to date have been noisy images in captchas, to train machine vision against them.

Adversarial.io seems to be just destructive, can’t you do something positive?

There has been a lot of dis­cus­sion about bias in AI, which leads to the mis­rep­re­sen­ta­tion of minori­ties, the fix­a­tion of the future in the past (because AI is trained on past events), the nor­ma­tive pow­er of describ­ing real­i­ty through attri­bu­tions and so on. 

We have seen ridi­coulous exam­ples where fail­ure rates of 40% were sold as a suc­cess (while a fail­ure rate of 0.5% would be accept­able), lots of false pos­i­tives in image recogn­tion and so on.

The pos­i­tive thing about adversarial.io is that it ques­tions the wrong assump­tion of »automa­tion is objec­tive«, and it sub­verts sys­tems, which should­n’t be in pro­duc­tive use at their cur­rent stage.

Doesn’t your project help closing existing loop holes?

Adversarial.io is an edu­ca­tion­al resource to inform about adver­sar­i­al tech­niques and make pub­lic what is cur­rent­ly still an expert discussion.

The big tech com­pa­nies under­take their own adver­sar­i­al research, com­pared to this we are a minor player.

What’s your tech?

We use open source soft­ware and stay inde­pen­dent from major tech companies.

Soft­ware: The front-end con­sists of a Word­Press Con­tent Man­age­ment Sys­tem, the back-end uses Python with the Flask frame­work and Pytorch. 

Hard­ware: A stan­dard off-the rack small serv­er with min­i­mal footprint.

We coor­di­nate through gitea, so if you want to join us, you can easily.