Building a smart garbage sorter using Rekognition along with inference
To conserve resources, we must recycle products like cardboards, clean plastic bottles, paper etc. We usually throw recyclable and non recyclable items in the same trash cans which have to be separated afterwards. If the separation is done at the early stage, it will be convenient. Some garbage dumping companies perform the task manually. Items can be recycled and used afterwards. The project I worked on is a garbage sorter which will be able to differentiate between recyclable and non-recyclable garbage items automatically. The project will help in sorting recycling products and reducing human power for separation purposes.
To implement my project, I used AWS Rekognition, an object detection technique. I trained my model 20 times with increasing number of images. Starting with 45 images, I ended up using 620 images. I started with simple images then increased to images with have little overlapping of objects to complex images. All of the images were resized to 416*416 pixels and were taken with a phone camera. I made sure the images were in range of 64*64 to 4096*4096 pixels as this is a requirement of Rekognition. My model can detect 10 different types of garbage (paper, can, tissue, cardboard, bottle, wrapper, plastic bag, wood, trash and compost). Compost class includes food wastes and trash class has items such as bubble sheets, batteries, medicine bottles, bulbs, cracked phones, plastic pieces etc .
I will show my project progress in other story.
METHODOLOGY
● Login to AWS account and navigate to Rekognition in console.
● Go and create a project. Give the project a unique name.
● Follow my Github post at the end of blog, from there download output-8.manifest file. Go to S3, create a bucket and upload the file. Now, go to the datasets and create a new dataset with a different name. Under ‘Image location’, select “Import images labeled by Amazon SageMaker Ground Truth”. Go to the S3 bucket and copy the path of output-8.manifest and paste it over there.
● Go to the project you created and hit “Train new model”. Select the dataset for training which you just created and for testing dataset, select “split dataset” option.
● When everything is set, start training the model. It might take upto an hour to train it.
● Once a model is trained, you can view your evaluation results, per label performance and test results. For my case, I got the final F1 score of 0.943.
● If you want to add more images, make sure images are resized to 416*416 pixels. For resizing, go to the github link and download “resizing-notebook”. Go to sagemaker, create a pytorch36 notebook and paste the code. Upload the image you want to resize in the notebook and give path of image in code. Run the cell and resized image will be save in the same folder where the notebook is saved. Make sure that when you resize a 2nd image, give it a different name otherwise it will overwrite it. On the same lines, you can also add augmented images to model. For that, go to GitHub link and use “augmentation-notebook”.
● When images are ready, go to the dataset which you created and add images in it. Start labeling them, it will make required bounding boxes.
● When the model is trained with new dataset, an updated manifest file will be available in S3.
● Start the model before using it. It should be in “running” status. For starting, stopping or detecting labels from models using CLI, go to the “commands notebook” on GitHub. Copy the start model command to start the model.
● For inference, take a new image and give it a path in the “inference notebook”. For that, create a new “conda_pytorch_p36” notebook and paste the code over there. You will get results as json format along with output image with bounding boxes.
These are some of the inference results.
LIVE DEMO
For live demo, I used the laptop camera. Put your garbage in front of the laptop camera and run the script. It will an capture image, store it locally, upload it to S3 bucket, call model in Rekognition, perform inference and give results as json output along with image with the bounding boxes and names of the labels.
For a live demo by accessing the laptop camera, go to file “demo-with-images.py”. Run the file in CLI using command “python demo-with-images.py”. Inference results will be displayed and also be saved locally.
Take a video yourself and add its local path in code “demo-with-video.py”. Run this python script. All frames from video and inferred frames are saved locally.
Go to “commands” file to run the 4th command to make video out of these frames.
Explanation of codes are added as comments in the files
For video results, view out2, out3 and out4.mp4 in GitHub post. This is how your videos will look like. All of my videos are at 25 fps.
GITHUB LINK:
LINK TO ACCESS IMAGES:
https://drive.google.com/drive/folders/1qleIpuyULfiy38wGp4ApFHH0J0ganRld?usp=share_link
