Projects | Swaroop Joshi

ISL Recognition and Learning

Thu, 04 Apr 2024 01:41:34 +0530

Indian Sign Language (ISL) is the primary mode of communication for the Deaf community in India. However, there is a lack of resources and tools for learning and recognizing ISL gestures. This project aims to address this gap by developing a large dataset of ISL gestures, implementing machine learning-based recognition systems, and creating game-based learning tools to facilitate the learning of ISL.

Collaborations:

Prof. Sougata Sen, Dept. of Computer Science and Information Systems, BITS Pilani, K K Birla Goa Campus
Prof. Surjya Ghosh, Dept. of Computer Science and Information Systems, BITS Pilani, K K Birla Goa Campus
Prof. Sravan Danda, Dept. of Computer Science and Information Systems, BITS Pilani, K K Birla Goa Campus
Prof. Thad Starner, Georgia Institute of Technology, USA

This project is supported by Google Inc. and DST-NSF.

Computational Thinking for Children with Disabilities

Sat, 04 Nov 2023 19:38:34 +0530

Computational Thinking (CT) is one of the 21st-century skills. It is considered crucial in determining the success and livelihood opportunities of the coming generations, just like literacy and maths were for the previous generations.

However, existing mainstream tools for teaching CT, such as Scratch, are not divyang-friendly. Unless we work on bridging this gap, we risk leaving these already underprivileged children further behind. The also emphasises on inclusive culture in schools, catering to the needs of all students, including those with disabilities.

In this project, we are working on designing and developing a set of game-based learning tools with tangible blocks and tablet apps that help children with certain disabilities develop computational thinking skills.

Collaborations:

Prof. Basavadatta Mitra, Dept. of Humanities and Social Sciences, BITS Pilani, K K Birla Goa Campus
Prof. Sujith Thomas, Dept. of Computer Science and Information Systems, BITS Pilani, K K Birla Goa Campus

This project is supported by BITS Pilani CDRF II.

Accessible Science Labs

Fri, 31 Mar 2023 16:01:28 +0530

This project is no longer active.

Background

The Rights of Persons with Disabilities Act, 2016 ( ) states that educational institutes should “provide reasonable accommodation according to the individual’s requirements and provide necessary support, individualised or otherwise, in environments that maximise academic and social development consistent with the goal of full inclusion” (Art. 16(iii-iv)). However, about 75% of children with disabilities (CWD) in India in their lifetime. CWDs are poorly represented in science, technology, engineering and mathematics (STEM) disciplines, partly because CWDs face access challenges to typical science labs in secondary, higher secondary and university settings. Access barriers may prevent students from gaining knowledge, demonstrating knowledge, and fully participating in lab activities.

We designed an Assistive Technology, SAlly (short for science accessibility or science ally), to address these challenges as part of the . It uses sensors to measure liquids in a beaker accurately. Thus, using the current prototype of SAlly, a visually impaired student can:

Identify various chemicals using accessible labels on the containers
Independently measure and pour out the required quantity of a liquid.

Ongoing work

We conducted user studies of our prototype with visually impaired students and their teachers in a school in Goa. Based on the feedback, we plan to refine the project to achieve these objectives:

Categorise the science experiments in Grades 6-12 in the NCERT curriculum so that similar experiments can be grouped and made accessible using the same technological innovation.
Modify and enhance the existing prototype to enable visually impaired students to independently pour out accurate volumes and identify a change in the colour of a chemical as a result of a reaction.
Create a lab manual that enables blind students to utilise our device to carry out experiments and for their educators to guide them.

We will evaluate existing sensors and smart devices that can identify changes in the colour of a liquid and incorporate the appropriate one in our prototype. We will then internally test the prototype with BITS students, followed by user testing with visually impaired students and their teachers in different contexts.

This project is supported by

Teaching Digital Accessibility through Serious Games

Sat, 20 Mar 2021 12:45:18 +0530

Background

The estimates that more than a billion people live with some form of disability. The mandates countries to “… promote the design, development, production, and distribution of accessible information and communications technologies and systems…”. To accomplish this, concepts and skills needed to develop accessible products and services must be included in computing education. Software companies like , , , and emphasize making their products accessible, and accessibility often features as a preferred skill in their job postings. For instance, a current for a Software Engineer position specifies “Experience developing accessible technologies” as a preferred qualification.

The Problem

Despite such increasing awareness in the software industry about accessibility in the recent years, many . Software professionals point to for this gap in implementing accessibility.

According to the , the main barriers to teaching accessibility include the lack of clear learning objectives about accessibility and their lack of knowledge about accessibility. Also, most instructors who teach accessibility have a background in human-computer interaction (HCI) and related fields and teach these topics in specialized courses. This indicates that other instructors may find it challenging to build their expertise and to incorporate accessibility topics in “mainstream” CS courses.

Our Contribution

We have developed fun, self-paced learning interventions, in the form of three sets of games:

Accessibility Awareness: (a) a special version of Wordle that asks you to guess a five-letter word, and tells you how it relates to accessibility, and (b) a game that asks you to burst the bubbles that carry a myth about accessibility, while saving the ones that carry a fact.
Access Quest: A role playing game that helps you learn WCAG guidelines about accessibility by solving puzzles and challenges a software developer may face in a day-to-day life.
Bug Hunter: A race against time game with a goal to find as many accessibility bugs in a web page as possible. The game is designed to assess your knowledge of WCAG guidelines.

You can try out these games at

We welcome your feedback and suggestion.

This project is supported by the

and BITS Pilani Research Initiation Grant.

CONSIDER

Wed, 04 Jan 2017 17:01:28 -0600

This project is no longer active.

This is part of my PhD Dissertation work.

Piaget’s classic work on cognitive development showed that engaging learners in critical discussions with peers about ideas that are different than theirs leads to deep conceptual understanding. Implementing such an approach in college level STEM (Science, Technology, Engineering, Math) courses has some specific challenges: (a) Short meeting times and large class sizes; (b) Competitive nature of the courses and single answer questions on assignments and exams; and (c) Overall lack of collaborative learning culture where students are unsure of how to seek help and many faculty members tend to think that engaging in collaborative activities may affect content coverage; etc. Based on Piaget’s theory, we have developed a highly innovative collaborative learning approach that exploits specific affordances of web technologies to address these challenges. This approach, named CONSIDER (short for CONflicting Ideas Discussed, Evaluated, and Resolved) allows creation of small groups of students with different ideas about the topic in question, engages them in a highly-structured rounds-based discussion so that the group progresses at an equitable pace, and makes their submissions anonymous to others so that students can receive the comments without any preconceived notions they may have about the poster.

We built a platform independent responsive web application using Google App Engine and Python to implement this approach and to compare it with the commonly used discussion forum approach, where students respond to other posts in a threaded discussion. This app was used in various undergraduate Computer Science & Engineering courses at The Ohio State University, where one discussion was conducted using the forum-based approach and the other using our approach in each course. Students performance on the pre- and post-discussion question and participation of individual students was measured.

Our evaluations are generally positive. See some for more details.

Geogame

Wed, 27 Apr 2016 17:01:23 -0600

This project is no longer active.

GeoGame is a web based role-playing game developed to let the students of geography experience the life of an ordinary farmer in developing countries, and thereby learn concepts of green revolution, etc. It is developed using MVC .NET architecture. I worked on this NSF-funded project (Grants and ) as a graduate research associate in .

For more information, read our book chapter “ ”.

gdfa

Tue, 05 Jan 2016 16:54:54 -0600

Note

This project is no longer active.

GDFA is a generic data flow analyzer for per function (i.e., intraprocedural) bit vector data flow analysis in GCC. I worked on this project between 2009 and 2011. at IIT Bombay maintains this project.