CanistAR by Intelligent System Information Society

AR projected images
CanistAR Logo

Please use the Google Drive link to download the app (APK file).

Inspiration

This project is inspired by a casual walk down Hosier Lane. Having encountered so many great graffiti artworks, we wondered if we can expand the space for more creativity and reduce any potential destructive impact on its surrounding environment.

What it does

It allows users to upload any pictures or drawings and 'paint' (by projecting using AR) them onto surfaces. The pictures or the drawings themselves can be rotated or resized and saved onto the virtual space of that surface. Other users can also 'see' the drawings and leave comments and likes.

The app effectively creates a virtual world, using AR, where users are able to put up drawings on any flat surface they can find.

How we built it

We leveraged Google's AR Core technologies in an android app, and Google's SceneForm SDK in order to project flat images onto real-world surfaces. In order to persist everyone's image with geospatial accuracy, we used Google's Cloud Anchor API and Firebase database to store information on every image. We also used Firebase authentication to authenticate users.

In order to filter out distressing images, we implemented a bespoke neural network designed to filter out violent imagery. The neural network is a Convolutional Autoencoder with 3 convolutional layers for the encoder and the decoder. We trained it on twelve thousand different images and were able to achieve 85% accuracy with our neural network.

Challenges we ran into

Convolutional Neural Network does not work as intended and we had to switch to Convolutional Autoencoder halfway through our development stage.

Because Google has discontinued the development of SceneForm SDK, which made it extremely difficult for us to use it as part of our app, as the documentation available is either outdated or incorrect. However, through much hard work and Googling, we eventually successfully integrated SceneForm.

Accomplishments that we're proud of

We grinded enough datasets with over 12,000 photos to train our AI model with over 85% accuracy.

Successfully learned how to use AR Core and Sceneform SDK to build a mobile AR app.

What we learned

Successfully learned how to use AR Core and Sceneform SDK to build a mobile AR app.

Researched new AI architectures.

What's next for Intelligent System Information Society

In the future, we want to expand CarnisAR to incorporate community features such as community interaction (commenting, voting, etc), and allow users to draw their own images using the app itself, instead of just uploading existing images.