Through the Lens
Through the Lens is a hybrid instrument that involves a piece of paper, a pen, and an OLED transparent display. My goal for this project is to understand the possibilities and the limitations of the device, and to come up with applications that are interesting, educational, and entertaining.
Bouncing Ball from Patt Vira
Steps:
- Draw inside the pre-calibrated section on a piece of paper (that is placed on top of a Wacom tablet) using a Wacom Inkling Sketch Pen.
- Place the OLED transparent display on top of the paper .
- Watch the graphics on the display interact with the drawings.
What did I learn?
From this project, I realized that I put a majority of my time trying to understand the basic use of the transparent display and how to get all the technology to work properly. Even though I wish I could have created more applications and presented my project beyond the proof of concept, I am now at the comfortable point where I can use the knowledge that I have to create interesting applications based on my own imagination and the feedback that I received from the outsiders perspective. The comments I received during both the final presentation and the show are invaluable. One important point I took away is that no one cares about the technology – what matters is the thing you do with the technology.
How can the project be improved?
The 4D System transparent display has a lot of potential, and I believe I have only tackled a small section of the possibilities. The feedback I received during the final presentation and the show is very helpful, and widen the scope of project ideas I can do with the knowledge I currently have. Here are the two directions I like to explore further.
1) Increase the area on a piece of paper to allow a bigger space for people to draw.
2) Use the display as a lens (think google glass)
I’d also like to get rid of the graphics tablet and make the display portable by exploring alternative ways of acquiring the pen strokes.
Acknowledgement:
- This project is inspired by Glassified by the Fluid Interfaces Group at the MIT Media Lab.
- Special thanks to Ali Momeni and Anirudh Sharma.
- Thanks to Golan Levin and the Frank-Ratchye STUDIO for Creative Inquiry grant for allowing me to purchase the 4D systems OLED transparent display.
My third milestone is to be able to make all the hardwares and softwares talk to each other properly, and have something interesting displayed on the screen. By this time, I am able to accomplish the task, sending the pen stroke data from Processing to 4D Systems through serial communication and have it displayed on the screen smoothly. I got a simple animation interacting with the pen strokes – a prove of concept that everything finally works together. Now, it’s time to make something interesting.
Bouncing Ball from Patt Vira
My original second milestone was to use computer vision as an alternate way to track the pen strokes. However, it took me longer to figure out how to send serial data from Processing to 4D Systems workshop, and how to draw properly on the screen. Therefore, the second week was spent mostly to figure out these problems.
I finally solved the problem from the first milestone, and was able to clear the screen after each draw function. The first video below shows a ball bouncing against two pre-drawn rectangular boundaries. The second one shows a real-time pen strokes being sent from the tablet to Processing to 4D Systems to the display.
Ball Bouncing Against Boundaries from Patt Vira on Vimeo.
Line Drawing from Patt Vira on Vimeo.
I found out from working more with the display that it unfortunately cannot handle heavy libraries, specifically box2D. Consequently, as a proof of concept, my goal is to draw some simple interaction. My next step is to be able to draw the lines from the Wacom tablet, and have the ball interacting with the these lines instead of the rectangular boundaries.
Through the Lens
Bouncing Ball from Patt Vira
Throughout the process, I have been able to get the different pieces of hardwares and the technology to talk and to work together properly. The video above shows a working prototype of a ball bouncing on the pre-written pen stroke. This is a prove of concept that everything works together. The next step is to explore other possibilities that puts meaning to the instrument through user interaction.
One application that I came up with is a maze game. The point of the game is to be able to control a ball (using LEFT, RIGHT, UP, and DOWN keys on the keyboard) along the path from the starting to the finishing points without touching a pre-drawn maze. As you can see from the series of images below, the maze can be customized to your likings.
This is one example of an application that adds meaning and interaction to the device. However, I am planning to create more applications as I become even more familiar with the instrument itself. I am also open to ideas and suggestions!
Below is a list of things needed to be done for this first milestone.
- Be familiar with 4D system workshop.
- Learn the limitations of the display: The kinds of data that can be displayed. Be familiar with the size.
- Figure out how to send data from the Wacom API to Processing
- Figure out how to send serial data from Processing to 4D system workshop.
- Laser cut the frame to house the display.
I did all the things I set out for myself for the first milestone. However, I need more time to play around with the screen to better understand its limitations. I laser cut a case to house the display, which is a very good idea because it is super fragile.
With a Processing built-in library, the communication between the Wacom tablet and Processing is easy to execute. It allows me to capture the pen strokes and send the data from the Wacom tablet to Processing in real time. As for the serial communication between Processing and 4D system workshop, I can now send serial data that draws basic shapes and writes texts on the screen. 4D system has a detailed documentation on serial communication that I can follow. However, without having any prior knowledge on serial communication, it did take me a while to understand how everything works. One problem I have right now is that I am unable to clear the screen after drawing a simple shape in order to do a basic animation (eg. a bouncing ball). There was some flickering on the display, and the program crashes.
1st milestone from Patt Vira
I think the initial step of getting basic animations to work on the display is very crucial. Once I have a better understanding of how to implement everything, I think creating applications will become much faster.
