Wadhwani AI

Publications

Filter:

All
AI/ML
Wadhwani AI: Annual Report 2022
For Wadhwani AI, 2022 has been a year of transformation. We met several goals: to actively and meaningfully engage with our ecosystem, including governments …

For Wadhwani AI, 2022 has been a year of transformation. We met several goals: to actively and meaningfully engage with our ecosystem, including governments at the central and state levels; to define problems holistically and with clarity; and to establish scalable, core competencies in solving problems rapidly using AI. We see these as mandatory building blocks for our mission of deploying our AI solutions at scale, as we aspire to positively impact 10 crore people in India with our solutions.

2023 will be an exciting year for us, and our focus will extend to spearheading the implementation of novel technologies in the realm of large language models and generative AI. 

Download the report to read about our work in 2022 and our vision for the future.

Download  

Share

BOLLWM: A real-world dataset for bollworm pest monitoring from cotton fields in India
This paper presents a dataset of agricultural pest images captured over five years by thousands of small holder farmers and farming extension workers across …

This paper presents a dataset of agricultural pest images captured over five years by thousands of small holder farmers and farming extension workers across India. The dataset has been used to support a mobile application that relies on artificial intelligence to assist farmers with pest management decisions. Creation came from a mix of organised data collection, and from mobile application usage that was less controlled. This makes the dataset unique within the pest detection community, exhibiting a number of characteristics that place it closer to other non-agricultural objected detection datasets. This not only makes the dataset applicable to future pest management applications, it opens the door for a wide variety of other research agendas.

View  
Download  

Share

CottonAce: Kharif 2021 Impact Report
CottonAce, Wadhwani AI’s early pest warning and advisory system, was developed to aid in the larger effort to improve the lives of cotton farmers …

CottonAce, Wadhwani AI’s early pest warning and advisory system, was developed to aid in the larger effort to improve the lives of cotton farmers in India. Between June to December 2021, the AI-powered solution was used by over 6,000 farmers across 60 districts and 10 states in the country.

Our latest report assesses the impact it has had on the ground, and outlines some of the challenges present in implementing an AI-powered pest management intervention in one of the most complex agricultural systems in the world. 

Download  

Share

A Case for Rejection in Low Resource ML Deployment
Building reliable AI decision support systems requires a robust set of data on which to train models; both with respect to quantity and diversity. …

Building reliable AI decision support systems requires a robust set of data on which to train models; both with respect to quantity and diversity. Obtaining such datasets can be difficult in resource limited settings, or for applications in early stages of deployment. Sample rejection is one way to work around this challenge, however much of the existing work in this area is ill-suited for such scenarios. This paper substantiates that position and proposes a simple solution as a proof of concept baseline.

View  
Download  

Share

COVID-19 Decision Support Playbook
This playbook by Wadhwani AI is a practical guide that summarises the work we did from a technical and public health perspective, to aid …

This playbook by Wadhwani AI is a practical guide that summarises the work we did from a technical and public health perspective, to aid government bodies towards their pandemic response through a combination of predictive modelling and data analytics.

It is meant for data science practitioners and epidemiologists, who may utilise our methodology and codebase in their respective research areas. Moreover, public health professionals and government officials may glean from the data pipelines, modelling framework and analytics capabilities to forecast disease spread in communities.

Download  

Share

AI Solutions for TB Elimination
The NTEP (National TB Elimination Program) along with USAID aims to aid innovators and leverage emerging technologies such as artificial intelligence (AI) to accelerate …

The NTEP (National TB Elimination Program) along with USAID aims to aid innovators and leverage emerging technologies such as artificial intelligence (AI) to accelerate tuberculosis elimination efforts. The NTEP has identified areas and articulated problems within the TB cascade of care that are AI-amenable. However, the NTEP is aware that this is a non-exhaustive list and is open to exploring new problem areas and solutions.

The purpose of this document is to motivate and invite individuals and organizations to develop AI-based solutions for TB. The conclusion section of the document describes the process for engaging with the NTEP and USAID on such solutions.

Download  

Share

Information Sufficiency via Fourier Expansion
We take an information-theoretic approach to identify nonlinear feature redundancies in unsupervised learning. We define a subset of features as sufficiently-informative when the joint …

We take an information-theoretic approach to identify nonlinear feature redundancies in unsupervised learning. We define a subset of features as sufficiently-informative when the joint entropy of all the input features equals to that of the chosen subset. We argue that the rest of the features are redundant as all the accessible information about the data can be captured from sufficiently-informative features. Next, instead of directly estimating the entropy, we propose a Fourier-based characterization. For that, we develop a novel Fourier expansion on the Boolean cube incorporating correlated random variables. This generalization of the standard Fourier analysis is beyond product probability spaces. Based on our Fourier framework, we propose a measure of redundancy for features in the unsupervised settings. We then, consider a variant of this measure with a search algorithm to reduce its computational complexity as low as with being the number of samples and the number of features. Besides the theoretical justifications, we test our method on various real-world and synthetic datasets. Our numerical results demonstrate that the proposed method outperforms state-of-the-art feature selection techniques.

View  
Download  

Share

Finding Relevant Information via a Discrete Fourier Expansion
A fundamental obstacle in learning information from data is the presence of nonlinear redundancies and dependencies in it. To address this, we propose a …

A fundamental obstacle in learning information from data is the presence of nonlinear redundancies and dependencies in it. To address this, we propose a Fourier-based approach to extract relevant information in the supervised setting. We first develop a novel Fourier expansion for functions of correlated binary random variables. This is a generalization of the standard Fourier expansion on the Boolean cube beyond product probability spaces. We further extend our Fourier analysis to stochastic mappings. As an important application of this analysis, we investigate learning with feature subset selection. We reformulate this problem in the Fourier domain, and introduce a computationally efficient measure for selecting features. Bridging the Bayesian error rate with the Fourier coefficients, we demonstrate that the Fourier expansion provides a powerful tool to characterize nonlinear dependencies in the features-label relation. Via theoretical analysis, we show that our proposed measure finds provably asymptotically optimal feature subsets. Lastly, we present an algorithm based on our measure and verify our findings via numerical experiments on various datasets.

View  
Download  

Share

Impact of data-splits on generalization: Identifying COVID-19 from cough and context
Rapidly scaling screening, testing and quarantine has shown to be an effective strategy to combat the COVID-19 pandemic. We consider the application of deep …

Rapidly scaling screening, testing and quarantine has shown to be an effective strategy to combat the COVID-19 pandemic. We consider the application of deep learning techniques to distinguish individuals with COVID from non-COVID by using data acquirable from a phone. Using cough and context (symptoms and meta-data) represent such a promising approach. Several independent works in this direction have shown promising results. However, none of them report performance across clinically relevant data splits. Specifically, the performance where the development and test sets are split in time (retrospective validation) and across sites (broad validation). Although there is meaningful generalization across these splits the performance significantly varies (up to 0.1 AUC score). In addition, we study the performance of symptomatic and asymptomatic individuals across these three splits. Finally, we show that our model focuses on meaningful features of the input, cough bouts for cough and relevant symptoms for context.

View  
Download  

Share

Interpretability of Epidemiological Models: The Curse of Non-Identifiability
Interpretability of epidemiological models is a key consideration, especially when these models are used in a public health setting. Interpretability is strongly linked to …

Interpretability of epidemiological models is a key consideration, especially when these models are used in a public health setting. Interpretability is strongly linked to the identifiability of the underlying model parameters, i.e., the ability to estimate parameter values with high confidence given observations. In this paper, we define three separate notions of identifiability that explore the different roles played by the model definition, the loss function, the fitting methodology, and the quality and quantity of data. We define an epidemiological compartmental model framework in which we highlight these non-identifiability issues and their mitigation.

View  
Download  

Share

Temporal Ordered Clustering in Dynamic Networks: Unsupervised and Semi-Supervised Learning Algorithms
In temporal ordered clustering , given a single snapshot of a dynamic network in which nodes arrive at distinct time instants, we aim at …

In temporal ordered clustering , given a single snapshot of a dynamic network in which nodes arrive at distinct time instants, we aim at partitioning its nodes into K ordered clusters C_1≺⋯≺C_K such that for i<j , nodes in cluster C_i arrived before nodes in cluster C_j , with K being a data-driven parameter and not known upfront. Such a problem is of considerable significance in many applications ranging from tracking the expansion of fake news to mapping the spread of information. We first formulate our problem for a general dynamic graph, and propose an integer programming framework that finds the optimal clustering, represented as a strict partial order set, achieving the best precision (i.e., fraction of successfully ordered node pairs) for a fixed density (i.e., fraction of comparable node pairs). We then develop a sequential importance procedure and design unsupervised and semi-supervised algorithms to find temporal ordered clusters that efficiently approximate the optimal solution. To illustrate the techniques, we apply our methods to the vertex copying (duplication-divergence) model which exhibits some edge-case challenges in inferring the clusters as compared to other network models. Finally, we validate the performance of the proposed algorithms on synthetic and real-world networks.

View  
Download  

Share

Wadhwani AI is a program of the AI Unit of the Lords Education and Health Society (LEHS)

Back to top ↑

Wadhwani AI is a program of the AI Unit
of National Entrepreneurship Network (NEN)

About Us

Our Approach
Core ML
Founder Donors
Our Team

Our Work

Pest Management
Tuberculosis
Maternal, Newborn, and Child Health
e-Health
Oral Reading Fluency

Knowledge Centre

Blog
GitHub
Publications
Newsletters
Press

Subscribe

Subscribe to receive updates

Careers

Open Positions
Careers
Working at Wadhwani AI

Partnerships

Our Partners
Collaborating With Us
Strategic Programs

Our Approach
Core ML
Founder Donors
Our Team

Pest Management
Tuberculosis
Maternal, Newborn, and Child Health
e-Health
Covid Decision Support
Cough Against Covid

Blog
GitHub
Publications
Newsletters
Press

Open Positions
Careers
Working at Wadhwani AI

Our Partners
Collaborating With Us
Strategic Programs

Contact Us

Facebook-f Twitter Youtube Linkedin-in Instagram

© 2024 Wadhwani AI

ML Engineer

ROLES AND RESPONSIBILITIES

An ML Engineer at Wadhwani AI will be responsible for building robust machine learning solutions to problems of societal importance; usually under the guidance of senior ML scientists, and in collaboration with dedicated software engineers. To our partners, a Wadhwani AI solution is generally a decision making tool that requires some piece of data to engage. It will be your responsibility to ensure that the information provided using that piece of data is sound. This not only requires robust learned models, but pipelines over which those models can be built, tweaked, tested, and monitored. The following subsections provide details from the perspective of solution design:

Early stage of proof of concept (PoC)

  • Setup and structure code bases that support an interactive ML experimentation process, as well as quick initial deployments
  • Develop and maintain toolsets and processes for ensuring the reproducibility of results
  • Code reviews with other technical team members at various stages of the PoC
  • Develop, extend, adopt a reliable, colab-like environment for ML

Late PoC

This is early to mid-stage of AI product development

  • Develop ETL pipelines. These can also be shared and/or owned by data engineers
  • Setup and maintain feature stores, databases, and data catalogs. Ensuring data veracity and lineage of on-demand pulls
  • Develop and support model health metrics

Post PoC

Responsibilities during production deployment

  • Develop and support A/B testing. Setup continuous integration and development (CI/CD) processes and pipelines for models
  • Develop and support continuous model monitoring
  • Define and publish service-level agreements (SLAs) for model serving. Such agreements include model latency, throughput, and reliability
  • L1/L2/L3 support for model debugging
  • Develop and support model serving environments
  • Model compression and distillation

We realize this list is broad and extensive. While the ideal candidate has some exposure to each of these topics, we also envision great candidates being experts at some subset. If either of those cases happens to be you, please apply.

DESIRED QUALIFICATIONS

Master’s degree or above in a STEM field. Several years of experience getting their hands dirty applying their craft.

Programming

  • Expert level Python programmer
  • Hands-on experience with Python libraries
    • Popular neural network libraries
    • Popular data science libraries (Pandas, numpy)
  • Knowledge of systems-level programming. Under the hood knowledge of C or C++
  • Experience and knowledge of various tools that fit into the model building pipeline. There are several – you should be able to speak to the pluses and minuses of a variety of tools given some challenge within the ML development pipeline
  • Database concepts; SQL
  • Experience with cloud platforms is a plus
mle

ML Scientist

ROLES AND RESPONSIBILITIES

As an ML Scientist at Wadhwani AI, you will be responsible for building robust machine learning solutions to problems of societal importance, usually under the guidance of senior ML scientists. You will participate in translating a problem in the social sector to a well-defined AI problem, in the development and execution of algorithms and solutions to the problem, in the successful and scaled deployment of the AI solution, and in defining appropriate metrics to evaluate the effectiveness of the deployed solution.

In order to apply machine learning for social good, you will need to understand user challenges and their context, curate and transform data, train and validate models, run simulations, and broadly derive insights from data. In doing so, you will work in cross-functional teams spanning ML modeling, engineering, product, and domain experts. You will also interface with social sector organizations as appropriate.  

REQUIREMENTS

Associate ML scientists will have a strong academic background in a quantitative field (see below) at the Bachelor’s or Master’s level, with project experience in applied machine learning. They will possess demonstrable skills in coding, data mining and analysis, and building and implementing ML or statistical models. Where needed, they will have to learn and adapt to the requirements imposed by real-life, scaled deployments. 

Candidates should have excellent communication skills and a willingness to adapt to the challenges of doing applied work for social good. 

DESIRED QUALIFICATIONS

  • B.Tech./B.E./B.S./M.Tech./M.E./M.S./M.Sc. or equivalent in Computer Science, Electrical Engineering, Statistics, Applied Mathematics, Physics, Economics, or a relevant quantitative field. Work experience beyond the terminal degree will determine the appropriate seniority level.
  • Solid software engineering skills across one or multiple languages including Python, C++, Java.
  • Interest in applying software engineering practices to ML projects.
  • Track record of project work in applied machine learning. Experience in applying AI models to concrete real-world problems is a plus.
  • Strong verbal and written communication skills in English.
mls