Notifications
MR SMAICS: Mixed Reality based Sangrahalya Margadarshak Artificial Intelligence ChatBot System
Showcase
271
0

INTRODUCTION

MR. SMAICS is an Android application developed using Unity for usage in museums and exhibitions about Mahatma Gandhi. It is a complete solution for guiding visitors about the exhibits and the gallery in the museum. It is equipped with Augmented Reality Mode and Virtual Reality Mode, each with its own specific functionally and specialty. This app makes use of all of the various utilities provided by Unity and the Asset Store community. It also utilizes a separate python production server application it uses for web scraping and Video Streaming.

FEATURES

It has the following functionalities corresponding to the modes available in the application prototype.

Augmented Reality mode:

  1. Static Image Description for exhibits.
  2. Web Scraping Image description
  3. Virtual Statue of Mahatma Gandhi.
  4. Augmented Reality Chatbot for asking more questions about the image.
  5. Augmented Reality Chatbot Web Scarper integration to get any general information.
  6. Augmented Reality Navigational System to go to Virtual Reality.
  7. Augmented Reality Chatbot Speech Recognition
  8. Augmented Reality Chatbot Vocabulary improvement to get meaning of any word.

Virtual Reality mode:

  1. Static Image Description for exhibits.
  2. Virtual Reality Chatbot for asking questions about the exhibits.
  3. Virtual Reality Chatbot Web Scarper integration to get any general information.
  4. Virtual Reality Navigational System to go to Augmented Reality Mode.
  5. Virtual Reality Chatbot Speech Recognition
  6. Virtual Reality Chatbot Vocabulary improvement to get meaning of any word.
  7. Virtual Reality Chatbot for loading video on the Virtual Screen.
  8. Virtual Reality Large Video Screen to see the Video of Mahatma Gandhi Childhood, Jalianwala Bagh, Salt Sathyagraha and Civil Disobedience Movements. (Streaming through the internet).

REQUIREMENTS

The following are requirements for the functioning of the application.
  • Augmented Reality Mode
  1. Android Phone With Android Version >5.0
  2. Active internet Connection
  • Virtual Reality Mode
  1. Android Phone With Android Version >5.0
  2. Active internet Connection
  3. Virtual Reality Headset
  4. Virtual Reality Remote (Optional)

INSTRUCTIONS

Augmented Reality Mode:

  1. If you want to know more about the exhibit/Gallery Photo, point your camera to get the description of the exhibit.
  2. Read the description on the Image appearing next to the exhibit
  3. If you want to know more information, click on the image to get more information
  4. Still want more information, There will a Mahatma Gandhi Statue. Click on it and talk to Mr. Smaics
  5. Ask Mr. Smaics to get more information through your voice
  6. If you want to get an immersive experience you can swipe on your screen or ask Mr. Smaics to take you to Virtual Reality.

Virtual Reality Mode:

  1. Make sure your phone is in Landscape orientation, make sure you have Virtual Reality Headset and Bluetooth Joystick and make sure that Virtual Reality works on your phone
  2. You can See Mr. Smaics as the robot in front of you
  3. You can talk to it and see the answer in a Virtual environment
  4. You can ask Mr. Smaics to Show you a movie
  5. Watch the movie in Virtual Reality mode in large screen
  6. If you want to know the meaning of work, you can ask Mr. Smaics

VOICE COMMANDS EXAMPLE:

Normal Conversation:
Hello How Are you? What Are you?
Meaning of word:
Tell me the Meaning of Sparse Meaning of Hope Tell me what Juice Means
Information about a topic:
Tell me about the Statue of Unity Tell me about Indira Gandhi Tell me about Rajesh Khanna Who is RajniKanth Who is Donald Trump
Go To VR mode:
Go to VR Mode Virtual Reality Mode Take me to Virtual Reality Mode
Video Changing:
Load Video about Civil Disobedience Show me Video about Salt Satyagraha

Future Developments:

Making the web scraped more dynamics as right now it can only scrape Wikipedia thus might return a wrong description.
Making the videos dynamic using a YouTube scraper and down-loader to get video for any topic along with serving it to unity
Improving the UI of the Application. and also Adding multi linguistic support to the app(Making it usable in hindi as well as other languages such as french or japanese)
Click Here to download the app.
Infant Sam Christian
Full Stack Dev, Solution Provider, Freelancer - Student
1
Comments