Current-Project | Swaroop Joshi

ISL Recognition and Learning

Thu, 04 Apr 2024 01:41:34 +0530

Indian Sign Language (ISL) is the primary mode of communication for the Deaf community in India. However, there is a lack of resources and tools for learning and recognizing ISL gestures. This project aims to address this gap by developing a large dataset of ISL gestures, implementing machine learning-based recognition systems, and creating game-based learning tools to facilitate the learning of ISL.

Collaborations:

Prof. Sougata Sen, Dept. of Computer Science and Information Systems, BITS Pilani, K K Birla Goa Campus
Prof. Surjya Ghosh, Dept. of Computer Science and Information Systems, BITS Pilani, K K Birla Goa Campus
Prof. Sravan Danda, Dept. of Computer Science and Information Systems, BITS Pilani, K K Birla Goa Campus
Prof. Thad Starner, Georgia Institute of Technology, USA

This project is supported by Google Inc. and DST-NSF.

Computational Thinking for Children with Disabilities

Sat, 04 Nov 2023 19:38:34 +0530

Computational Thinking (CT) is one of the 21st-century skills. It is considered crucial in determining the success and livelihood opportunities of the coming generations, just like literacy and maths were for the previous generations.

However, existing mainstream tools for teaching CT, such as Scratch, are not divyang-friendly. Unless we work on bridging this gap, we risk leaving these already underprivileged children further behind. The also emphasises on inclusive culture in schools, catering to the needs of all students, including those with disabilities.

In this project, we are working on designing and developing a set of game-based learning tools with tangible blocks and tablet apps that help children with certain disabilities develop computational thinking skills.

Collaborations:

Prof. Basavadatta Mitra, Dept. of Humanities and Social Sciences, BITS Pilani, K K Birla Goa Campus
Prof. Sujith Thomas, Dept. of Computer Science and Information Systems, BITS Pilani, K K Birla Goa Campus

This project is supported by BITS Pilani CDRF II.

Teaching Digital Accessibility through Serious Games

Sat, 20 Mar 2021 12:45:18 +0530

Background

The estimates that more than a billion people live with some form of disability. The mandates countries to “… promote the design, development, production, and distribution of accessible information and communications technologies and systems…”. To accomplish this, concepts and skills needed to develop accessible products and services must be included in computing education. Software companies like , , , and emphasize making their products accessible, and accessibility often features as a preferred skill in their job postings. For instance, a current for a Software Engineer position specifies “Experience developing accessible technologies” as a preferred qualification.

The Problem

Despite such increasing awareness in the software industry about accessibility in the recent years, many . Software professionals point to for this gap in implementing accessibility.

According to the , the main barriers to teaching accessibility include the lack of clear learning objectives about accessibility and their lack of knowledge about accessibility. Also, most instructors who teach accessibility have a background in human-computer interaction (HCI) and related fields and teach these topics in specialized courses. This indicates that other instructors may find it challenging to build their expertise and to incorporate accessibility topics in “mainstream” CS courses.

Our Contribution

We have developed fun, self-paced learning interventions, in the form of three sets of games:

Accessibility Awareness: (a) a special version of Wordle that asks you to guess a five-letter word, and tells you how it relates to accessibility, and (b) a game that asks you to burst the bubbles that carry a myth about accessibility, while saving the ones that carry a fact.
Access Quest: A role playing game that helps you learn WCAG guidelines about accessibility by solving puzzles and challenges a software developer may face in a day-to-day life.
Bug Hunter: A race against time game with a goal to find as many accessibility bugs in a web page as possible. The game is designed to assess your knowledge of WCAG guidelines.

You can try out these games at

We welcome your feedback and suggestion.

This project is supported by the

and BITS Pilani Research Initiation Grant.