WIDS-2024-Can-Computers-Think

This Winter in Data Science Project under the Analytics Club, IIT Bombay is being mentored by Nirav Bhattad and me

Contents

• WEEK 0
  • Resources for Git
  • Resources for Python
  • Resources for Numpy
  • Resources for Pandas
  • Resources for Matplotlib
  • Combined Resources
• WEEK 1
• WEEK 2
• WEEK 3

This project focuses on introducing you to the world of Reinforcement Learning by solving puzzles (and interacting with us....) in the most fun way possible!!!!

Week 0:

In the Week 0, we want you to learn Python (Python is a good relief if you're bored of C++ from CS 101 this sem) and modules like numpy, pandas and matplotlib (They are the goto libraries and the first imported when you're starting with any DS project, so it's good to have a grasp on them)

Choose whatever resource suits you and get acquainted with the modules to move forward with the project.

Resources for Git

I'd love it if you also learn git for working with projects (it's very useful for version control and team integration)

This is a good lecture on git for beginners

Git - https://youtu.be/NcoBAfJ6l2Q?si=FRLMmKKwj01E8ePw

Resources for Python

This is a good video but long, should keep you occupied for a bit.

Programming with Mosh: Youtube

Resources for Numpy

Numpy Tutorial - Numpy Official Website

Resources for Pandas

Pandas Tutorial - https://youtu.be/vmEHCJofslg?si=1efKYTqQRdDECaNE

Official Tutorials - https://pandas.pydata.org/pandas-docs/version/0.15/tutorials.html

Resources for Matplotlib

You can just look up what plot you want at that moment for matplotlib or

Official Tutorial - https://matplotlib.org/stable/tutorials/index.html

Combined Resources

Numpy, Pandas and Matplotlib in a Playlist - https://youtube.com/playlist?list=PL9n0l8rSshSnragNblKDBsT8Xu3otp3jA&si=1zoqrRjvGLP4QMmU

Some Python Notebooks for your practice can be found in the Week 0 Folder

Week 1:

In this week, you're gonna work on the Multi-Bandit Problem from Sutton and Barto (available in the resources folder)

You're going to implement the greedy and $\epsilon$-greedy algorithms and observe how one is better than the other and make changes

Read the Chapter in Sutton and Barto and implement the algorithms in the work_on_bandits.ipynb file

Week 2:

• The assignment in Assignment_2.ipynb is about simple modelling of some basic RL environment which is to help you understand the basic working of MDPs. Note that most problems we work with using RL always have a MDP working under the hood, which we may or may not(most of the times) know. We will explore this in later weeks

• The second is an optional reading assignment, here which you can read if you intreseted in understanding and modelling of Markov chains in detail. Note that it requires some basic understanding of probability, expected value and ranodom variables so its ok if you don't understand in the beginning. Although it is important that you google these terms to get a baisc idea if you don't know them , as it will be used frequently in RL.

Week 3:

• Do the weeks matter anymore?

• The assignments in the folders Assignments and More Assignments are to be done in this week. The first one is a simple DP problem and the second one is a simple MDP problem.

• Make sure you have fun while doing them and don't restrict yourself to just solving the problem but try exploring the problem and the solution in depth.

• I'm sad nobody found the Miguel Morales reference! :(