HMP

Problem

• Malnutrition remains widespread worldwide.
• 1/3 of African children are stunted.
• Anthropometric measurements are essential to diagnose malnutrition & other health conditions.
• Measurements taken on site with manual means are often imprecise/unreliable, especially with children. The idea is to use advances in machine learning and computer vision to obtain these measurements purely from images taken using mobile phones.

Solution

• A mobile application based on Ai that can estimate body measures from a cellphone image.
• A sustainable business model for deployment in both developed and developing countries.

Impact

• Fight malnutrition in low income contexts
• Innovative medical diagnoses based on Ai-enabled anthropometry.
• Dual business models: premium in industrial context vs freemium in low-income contexts.

Progress

The goal of the project is to estimate the height and weight of an individual from a frontal and lateral picture of an individual taken in real-life setting using an uncalibrated mobile phone camera. This is a very challenging problem from the point of view of computer vision. Two approaches were taken to work towards a solution.

The first approach involved the estimation of height followed by the estimation of weight in two steps. In order to estimate the height, the joint locations of limbs were detected using a method available in existing literature. The distances between joints were used to estimate the full height of the individual based on similar annotated data. In order to detect the weight, first the silhouette of the individual is detected using an existing method. Assuming a near elliptical shape and given the height, the volume of an individual is estimated using the frontal and lateral silhouettes as the cross-sectional measurements. Density values are regressed upon for each pixel-level slice of the estimated volume, using which the weight can be estimated.

The main drawback of the two-step process is that an error in the computation of the height accumulates with the error error in the estimation of the weight. To avoid this, a second, single step approach is taken. A deep network is trained to simultaneously compute the silhouette, the skeletal joints, as well as the attributes of height and weight in a single step, taking only a frontal image as input (see figure below). A paper presenting this was a to accepted in ICASSP 2020.

Since the images and data related to the child images were small in number and privacy sensitive, all training was performed using publicly available data for adults. The idea is to perform the training on these and then fine-tune the models for the child images. The result was that the model attained acceptable error (less than 10%) in predicting height, but more that that for weight estimation.