Varun Raveendra

What is life in Amherst like?

Mon, 28 Mar 2022 03:30:55 +0000

It’s been nearly three and a half years since I started my PhD at UMass Amherst. Having spent my entire life in huge metropolis cities, I was curious to know what life in the small college town of Amherst, Massachusetts and Pioneer Valley would be like. Over the years, I have slowly fallen in love with the place. Through this blog, I will try to illustrate what I’ve liked best about the area and my university. I will also share some difficulties I faced, and why they are not as bad as one might think.

Disclaimer — This blog solely reflects my opinion. Depending on one’s individual circumstances and perspectives (mine are listed below), one’s opinions about life in the area can be very different. Emery Berger’s blog as a very positive outlook on about life in the area, while Katherine Thai’s blog is a more negative take on life in the area, but a very positive outlook on our PhD program!

My personal circumstances — I came to Amherst for my PhD in Computer Science straight after my undergraduate program in India. I didn’t know how to drive when I arrived here, but loved biking. Besides Amherst, I’ve lived (2 months or more) in New York City, Chicago, Pune, Mumbai. I have also visited several other places internationally.

Advantages of living in Amherst

Safety and Cosmopolitan Environment — Amherst is by far the safest US city / town I have lived in, crime has almost been non-existent in my 3.5 years here (compared to daily crime alerts on many other university campuses and big US cities). I have never once felt unsafe biking late at night in the UMass area. University faculty, students, staff and retired professionals make up a large fraction of the residential body, and people are highly educated and liberal (Emery Berger’s blog says 42% people have a graduate degree, nearly highest in America). A lab alumni has said that even strangers on the bus can help them with your research! This is a perfect place to focus on your research without distractions. The area is not at all crowded, which means no traffic jams and safety during pandemics. The area is also very LGBTQ friendly, see the Wikipedia section on this topic for detailed statistics. Note that places north of the mountains (Amherst, Northampton, Hadley, Sunderland) are much closer to campus and safer than Holyoke / Springfield.
Proximity to nature — Amherst is full of natural beauty. There are gorgeous mountains, lakes and forests all over the area (even on UMass campus — a beautiful pond and a forested hill!) There are several mountains to hike, trails to bike and gentle nature walks all within 10-15 minutes from UMass! I have really enjoyed biking / hiking the area — enjoying nature is such an excellent break from research. Amherst is also where I have discovered my passion for bird watching and photography (my birding page), which is my favourite outdoor activity. I doubt I would have discovered my love for nature had I gone to a big city for graduate studies.
Affordable cost of living — Amherst is very affordable to live in compared to bigger US cities, especially monthly rent. Moreover, UMass has healthy graduate stipends and a strong graduate student union. The affordable cost of living allows you to pursue several hobbies which may be difficult on tight budgets (bird watching can be expensive due to camera equipment).
Variety of Food and Groceries — Despite being a small college town, the area is full of excellent restaurants with a wide variety of cuisines from across the world (Emery Berger’s blog has many more on coffee shops, pizza, restaurants in the area). UMass has been named the best dining program in USA for five consecutive years according to Princeton Review (details). I can personally vouch for it — my parents while visiting USA prefered UMass Dining halls over restaurants in NYC, SF, LA, LV! Due to numerous local farms, there are lots of good options for locally produced groceries. Several farms offer activities like picking strawberries, apples, and plums in the summer seasons. The area also has very good international grocery stores — Indian, Chinese, Japanese, Korean you name it! Large shopping complexes for most of the popular US brands are also here / within 20 minutes drive.
Active Communities — Over the years I’ve grown to appreciate the community of people living here who share common interests. It’s usually easy to find on-campus / off-campus communities of people for several hobbies like sports, rock climbing, karate, reading clubs, contra dancing, wine tasting, South Asian / Jewish / Chinese communities etc. I have personally had a great time with the talented bird watching community in the area. We often go on bird watching walks together, share sightings, and get together for social events!
Unique places to see — Western Massachusetts is not New York City, but it is unique nevertheless. Besides natural beauty, it is full of unique man-made places — Mass MoCA, a crazy hairpin with beautiful views, an art museum made of garbage scraps, Emily Dickinson’s home, the tallest academic library in the world with amazing views, a falconry, a butterfly conservatory, Yankees’ Candles’ wonderful headquarters, a large peace pagoda, Satan’s kingdom, one of the largest unfiltered water supplies in the United States with a fantastic nature reserve surrounding it, a children’s book museum, basketball / volleyball hall of fame, dinosaur museum and footprints, unique bridges, the largest tree on the East Side of Missisippi, numerous farms with mazes / strawberry picking / mega-rare birds, an accessible summit house with stunning views right up to Vermont and Connecticut, interesting libraries and bookmills, several hydroelectric projects on the Connecticut River, self-sufficient houses and so much more. The area also has excellent movie theaters, a zoo, an Six Flags amusement park, casino, go-karting tracks, rock-climbing gym, escape rooms, laser tag, mini-golf, ice-skating rinks, numerous bars, contra dancing club, skiing resorts, flight school, water sport activities (on Connecticut River) etc., Amherst is conveniently located with regular bus services to Northampton (20 minutes away), Springfield (40 minutes), Boston (2 hrs away), New York City (3.5 hrs away). It is 1-3 hrs from Vermont, New Hampshire (Fall Foliage!), the beautiful Massachusetts coast (Plum Island, Gloucester, Cape Ann), Connecticut coast (New Haven, Stratford, Bridgeport), Rhode Island (Providence, New Port) and Cape Cod (Provincetown, Martha’s Vineyard).

(TODO — points about CICS, UMass, UMass NLP)

Difficulties of living in Amherst

Having a car can enhance your Amherst experience — For the first two years I did not have a car, I would bike everywhere or take the bus. While I still enjoyed myself on bike trails and had a comfortable life, buying a car significantly improved my Amherst experience. I visited most of the unique places I listed above after I bought a car. It also enabled me to bird watch and hike a lot more (at the expense of biking). Grocery shopping also became a lot simpler — rather than renting cars, biking or taking the bus, I could simply drive end-to-end. That being said, life without a car is very much possible. Public transport between residential areas and UMass is excellent and free of cost, there are several car rental spots right on campus (we have a driving school in Amherst too!). Grocery, food and Amazon delivery in the area is very good, reducing the need to drive from place to place (at the expense of some flexibility).
The weather is not for everyone — Amherst, like several places in USA, gets all four seasons. Beautiful Fall and Spring season with colourful trees and pleasant weather, hot summers and cold snowy winters. A few people love the diversity in seasons, some others hate the cold (myself included). The cold weather does open up several winter activities — there’s an active community of UMass students who go to nearby resorts (within 40min - 1hr drive) for skiing; UMass has one of the best ice hockey college teams in USA; I recently discovered how amazing winter hiking is! One aspect of New England weather is the large variation in temperature from day-to-day and Noreaster storms. On the bright side, it is a meterologist / weather geek’s paradise — the region has one of the highest concentration of weather geeks (book on New England weather). Personally, I enjoy following meterologists on Twitter as they talk about the unpredictable New England weather. Biking around does get hard in the winter, but the area is bikeable on most winter days (except 4-5 snow days a year) with the right winter clothing.
Amherst is not a big city — It does not have an airport (nearest domestic airport is 50 min away, nearest international airports are in Boston / NYC), it has lesser activity / things to do / concerts than a place like NYC, Ubers can be hard to get. But there is still a LOT to do, and the Pioneer Valley has lot more to offer than big cities in terms of outdoor activities (as I have described in the “Pros” section above). Besides, if you like indoor activities, life in Amherst is no different from life in a big city (yes, we do have good internet and fast delivery services).

If you like a rural (but still fairly cosmopolitan) setting, clean air, a calm atmosphere, natural beauty and a student town vibe, Amherst is the place to be!

Few-shot Controllable Style Transfer for Low-Resource Multilingual Settings - Project Page

Thu, 03 Mar 2022 03:30:55 +0000

This is a project page for our ACL 2022 paper on few-shot multilingual style transfer. For more details, contact me at kalpesh@cs.umass.edu.

arXiv: https://arxiv.org/abs/2110.07385
Evaluation Dataset: https://github.com/google-research-datasets/informal (formality transfer in 4 Indic languages hi, bn, kn, te)
slides: link
video: link

Hurdles to Progress in Long-form Question Answering - Project Page

Thu, 06 May 2021 03:30:55 +0000

This is a project page for our NAACL 2021 paper on long-form question answering. For more details, contact me at kalpesh@cs.umass.edu.

Abstract: The task of long-form question answering (LFQA) involves retrieving documents relevant to a given question and using them to generate a paragraph-length answer. While many models have recently been proposed for LFQA, we show in this paper that the task formulation raises fundamental challenges regarding evaluation and dataset creation that currently preclude meaningful modeling progress. To demonstrate these challenges, we first design a new system that relies on sparse attention and contrastive retriever learning to achieve state-of-the-art performance on the ELI5 LFQA dataset. While our system tops the public leaderboard, a detailed analysis reveals several troubling trends: (1) our system’s generated answers are not actually grounded in the documents that it retrieves; (2) ELI5 contains significant train / test overlap, as at least 81% of ELI5 validation questions occur in paraphrased form in the training set; (3) ROUGE-L is not an informative metric of generated answer quality and can be easily gamed; and (4) human evaluations used for other text generation tasks are unreliable for LFQA. We provide suggestions to mitigate each of these issues, which we hope will lead to more rigorous LFQA research and meaningful progress in the future.

arXiv: https://arxiv.org/abs/2103.06332
blogpost: https://ai.googleblog.com/2021/03/progress-and-challenges-in-long-form.html
slides: https://docs.google.com/presentation/d/1kkl0fGbhEqWnUDkcSbFsDWIKnojlR_HFiCvhAhXW2Uk/edit?usp=sharing
tweet: https://twitter.com/kalpeshk2011/status/1374443466537639939
video: https://drive.google.com/file/d/1OnArDF9tUsjDM29CI7seCbtnsCWnOkVg/view?usp=sharing

code: https://github.com/martiansideofthemoon/hurdles-longform-qa (supports inference from pretrained generator / retriever models, evaluation + analysis scripts, details in README file)
original Routing Transformer codebase: https://github.com/google-research/google-research/tree/master/routing_transformer

external summaries: Ruder’s newsletter, video1, video2, VentureBeat, SearchEngineJournal, MarkTechPost, TechStory

Model Extraction Attacks on BERT - Project Page

Sat, 04 Apr 2020 03:30:55 +0000

This is a project page for our ICLR 2020 paper on model extraction attacks on BERT. For more details, contact me at kalpesh@cs.umass.edu.

Thieves on Sesame Street! Model Extraction of BERT-based APIs
Kalpesh Krishna, Gaurav Singh Tomar, Ankur P. Parikh, Nicolas Papernot, Mohit Iyyer

arXiv: https://arxiv.org/abs/1910.12366
OpenReview: https://openreview.net/forum?id=Byl5NREFDr

blogpost: http://www.cleverhans.io/2020/04/06/stealing-bert.html
Twitter: https://twitter.com/kalpeshk2011/status/1247550470454984705

codebase: https://github.com/google-research/language/tree/master/language/bert_extraction
slide deck: https://docs.google.com/presentation/d/1emJb07ER8Gje6PGauIU–t0DCgvECJlnjVQfv9rOUkM/edit?usp=sharing
ICLR presentation & chat: https://iclr.cc/virtual_2020/poster_Byl5NREFDr.html

external coverage: YouTube summary by TechViz // NLPHighlights Podcast // Sebastian Ruder’s NLP News // HubofML Newsletter // TopBot’s Best NLP Papers at ICLR 2020 // VentureBeat ICLR 2020 Cover Photo // MadeWithML

Student Perspectives on Applying to NLP PhD Programs

Thu, 24 Oct 2019 03:30:55 +0000

Recently, we compiled a survey of NLP PhD applicants on their graduate school application experience. Our survey can be found here!

This post was written by: Akari Asai, John Hewitt, Sidd Karamcheti, Kalpesh Krishna, Nelson Liu, Roma Patel, and Nicholas Tomlin.

Thanks to our amazing survey respondents: Akari Asai, Aishwarya Kamath, Sidd Karamcheti, Kalpesh Krishna, Lucy Li, Kevin Lin, Nelson Liu, Sabrina Mielke, Roma Patel, Nicholas Tomlin, Eric Wallace, and Michihiro Yasunaga.

Going for a PhD - Insight University Series

Sat, 05 Oct 2019 03:30:55 +0000

I recently wrote a blog for Insight, IIT Bombay’s student media body describing my decision to pursue a PhD in natural language processing, the graduate application process and life at UMass Amherst.

You can find the blog here!

Crowdsourcing 101

Tue, 05 Jun 2018 03:30:55 +0000

I recently used crowdsourced experiments for the first time during my Bachelor’s Thesis Project with Prof. Preethi Jyothi. In this blog I will share a few important steps we took to improve the quality of our crowdsourced data. I sincerely thank Prof. Mohit Iyyer for suggesting several useful tips during these experiments, many of which will appear throughout this blog.

Introduction

Google defines crowdsourcing as “the practice of obtaining information or input into a task or project by enlisting the services of a large number of people, either paid or unpaid, typically via the Internet”. With the advent of deep learning, it is getting increasingly important to create high-quality large-scale datasets. This is where crowdsourcing can play an important role.

Crowdsourcing is very common in companies having products with a large userbase. For instance, Google deploys crowdsourcing on a very large scale for obtaining live traffic updates. Facebook can potentially use Messenger conversations for intelligent dialog agents.

Services

Unfortunately, not every organization plays an important role in the day-to-day lives of millions of people. Sometimes, even companies like Google might need very specific datasets which cannot be obtained directly using their products. This is when you turn to dedicated crowdsourcing services where crowdworkers are paid for performing certain tasks. Two popular online crowdsourcing services are Amazon Mechanical Turk and Figure Eight (which was previously known as CrowdFlower). For the result of this article, we will focus on Figure Eight.

Task Setup

In the Figure Eight interface, you begin by adding a number of data rows, often CSV data (in the “DATA” tab). Make sure the first row in the file consists of column titles. Here are Figure Eight’s guidelines on adding data.

The next stage (“DESIGN” tab) requires you to design the actual task. Crowdsourced experiments need to be well-thought of from the crowdworker’s perspective. Good experiments have clear instructions and less ambiguity. MCQ questions are generally preferred over textboxes for more meaningful data collection. For instance, let’s focus on the task of one sentence text classification. If we are specifically interested in sentiment, we should have options like “positive”, “negative”, “neutral” rather than leave a textbox and expect the crowdworker to fill in the sentiment. A person might use positive words “happy”, “joyful” etc. to describe the sentiment instead. For the rest of the article, we will focus on a binary sentiment classification task of one sentence.

The instructions section is quite important, and you could expect most crowdworkers to go through it atleast once during the task. Here are a few tips I learnt from my experience -

Try to make the title and instructions as unbiased as possible. Sentences like “most of these sentences are unlikely to be positive” should be avoided.
Instructions should suitably describe the task to be performed. One way to ensure this is to show the task preview to a third person unfamiliar with the research project or problem setting.
Try to include examples for every possible answer. For instance, in a binary sentiment classification task sample positive, negative and neutral sentences should be presented.
A “Tips” section should be added to describe unclear questions or options. For instance, pre-processing details should be described if they are significantly going to affect the question statement.
Use HTML formatting (colors, bolds, italics, underline)

Here is the instruction set I used during my experiments.

It is a good idea to read Figure Eight’s official Design Tips and Instruction Tips.

Quality Assurance

The next tab (“Quality”) is a quality control measure to ensure that only those people who understand the task correctly are recording their judgements. Every crowdworker appears for a test before starting the task. Throughout the task, the worker is presented with a few hidden test questions. These tests are designed by us, along with a gold set of correct answers (and an option to choose multiple correct answers for ambiguous questions). The crowdworker’s judgements on these test questions determine their competence for the task. Figure Eight rejects workers who fail the initial quiz and marks a user’s judgements as “Untrusted” if he makes several mistakes on subsequent hidden test questions. Figure Eight requires you to create atleast 8 test questions and often recommends you to create more test questions to speed up data collection. Figure Eight has released a couple of guides on Test Questions Best Practices and Monitoring Test Questions.

In my experience, test questions should be very close to the actual task, but as less ambiguous as possible. Ambiguous questions are often contested by crowdworkers and might affect the quality of judgements provided. Very easy test questions are also harmful, since they might be passed by spammers or incompetent crowdworkers (a big problem!). Finally, it’s also a good idea to keep an even distribution of answers so prevent biasing crowdworkers while they perform the actual task.

Keeping good test questions isn’t the only quality assurance technique (and often not enough). A few additional settings can often improve the data collected significantly. Here are a few such settings,

Contributor Channels - Keep this at Level 3 if time is not a constraint, but switch to Level 2 if you are in a rush. Level 2 and 3 crowdworkers have a reputation of getting test questions correct (based on their contribution history on Figure Eight) and are less likely to be spammers. Advanced settings include choosing contributor channels (Figure Eight has a guide on this) and language restrictions.
Geography - It might help allowing crowdworkers from certain locations only for your experiments. For instance in complicated English NLP tasks, it is a good idea to allow workers only from predominantly English speaking countries. For regional tasks, it makes no sense allowing workers not residing in that region.
Quality Control Settings - This setting allows you to keep a minimum time per page (to remove clear spammers) and maximum judgements per worker (to further reduce the effect of bad workers who might slip through). Figure Eight has published a guide for quality control settings.

Launch & Payment

Finally, jobs can be launched in the “Launch” tab on Figure Eight. Figure Eight recommends launching a part of the data (typically 100 rows) as a test run before launching the rest of the data. The “Judgements per Row” setting is again very experiment-specific. Figure Eight offers a “Dynamic Judgements” option, an automated solution to evaluate rows with a less agreement multiple times. (here is their guide)

A very important section here is the price per judgement. It is recommended to pay atleast 7.25$ per hour (minimum wage in USA). One way to convert this to “Price per Judgement” is to perform the task yourself and assess the time it takes per judgement. Generally, a larger payment will cost you significantly more but will speed up the data collection. The difficulty of the task should also be considered. Crowdworkers tend to prefer easier, repetitive tasks (like sentiment classification) rather than more challenging tasks requiring mental effort (like summary writing). Be prepared to pay higher wages to convince crowdworkers that the extra effort is worth it!

Figure Eight has a Jobs Cost FAQ aricle for reference.

Monitoring & Speed

Now that the job has been launched, you can sit back and enjoy the magic of crowdsourcing! The Figure Eight interface is kind-of addicting and you can monitor indvidual contributors as well as assess the overall progress. It is important to keep track of the test questions many workers are marking incorrectly, since they might be ambiguous or mistakes on your part. Workers might contest test questions and you can forgive workers who present a valid explanation for their answers. Finally, some workers might fill a job feedback survey, which is an assessment of the ease, clarity and payment in the job.

Jobs might not run as fast as you need them to. Here are a few useful tricks to speed up your jobs,

Increase Payment - This is the most reliable way to speed up data collection. However, I could not find a way to increase the payment for data rows that were in progress. Maybe it is a better idea not to order all data rows at once.
Increase Judgements Per User - This is a helpful trick but might reduce the quality of data collected. This setting can eadjusted under the “Quality Control” setting tab.
Contributor Channels - Reducing the minimum contributor level (Level 3 to Level 2 for instance) or making the geographic settings more liberal will certainly speed up jobs. Again do this with care to not compromise too much on the quality.
Internal Workforce - Yes, you can ask your friends to perform the task too, free of cost! While the maximum judgements per user limit would still apply, this is a very reliable way to speed up your crowdsourced experiments. You won’t compromise on quality either, since internal workers need to pass the test too and I am sure you would choose your friends wisely! :)

Figure Eight has several guides on monitoring jobs, a Dashboard Guide, on Auto-Paused Jobs and a guide to convert finalized rows into test questions.

Results

Once jobs complete, there are several detailed reports available for download. (under the “Results” tab). Make sure you adjust the settings to download data in the most suitable format for post-processing. Figure Eight has a nice guide on reports.

I have had a lot of fun in my first crowdsourcing experiment, and I hope to continue doing such experiments during my PhD. If you would like to add something to this article, feel free to get in touch!

Computer Science Opportunities

Mon, 04 Jun 2018 03:30:55 +0000

This is an article enlisting a number of opportunities in Computer Science you could pursue as an undergraduate at IIT Bombay. It does not talk about career paths in Computer Science after graduating, rather focusses on what you can do while you are an undergraduate at IIT Bombay.

Courses

Computer Science Minor Degree - A suite of five basic courses in Computer Science designed for non-CS undergraduates. Some of these courses are simpler than their counterparts for CS undergraduates, but they create a good foundation for Computer Science studies. You can find more information about the courses in the Course Information Booklet. I have also written a short review on Page 24 in the Sophomore Booklet.
An RnD project (CS490) or a 6xx / 7xx CS course could be included in the minor degree along with four minor courses.
Computer Science Coursework - In addition to the minor courses, you are free to take CS courses as your Institute Electives or ALCs (Additional Learning Courses). While this might need instructor approval occasionally, it is generally quite smooth for the elective courses (4xx, 6xx, 7xx). I don’t recommend sitting-through courses (attending classes only), unless you are looking to learn a very specific topic or part of the course, since you are unlikely to appreciate the nitty-gritties of a subject without regular practice, assignments and examinations.
Your Own Department - Several faculty in non-CSE department work in areas closely related to Computer Science. In turn, they conduct courses in their own department at the intersection of CS and their own department. Notably, the EE department has active courses in cryptography, information theory, computer vision, machine learning, computer architecture etc. These electives could be taken up as Department Electives or ALCs (Additional Learning Courses).

Research Opportunities

Computer Science Faculty - Several CS faculty are actively looking for students to work on research projects with them. These projects are not restricted to the CS students, and with the right background and motivation faculty are happy to work with non-CS students as well. Here is a complete list of faculty at CSE IIT Bombay. It’s a good idea to work with faculty via Seminars, RnD Projects (CS490), or BTech Projects in the CS department since you obtain course credits for your work and it’s generally easier to score good grades in these projects. (Crediting the projects will also motivate you to work harder!)
Your Own Department - Several faculty work at the intersection on CS and their home department, and working with them is a good idea for CS experience. For instance, Prof. Prabhu Ramachandran at Aerospace, IIT Bombay works in computation fluid dynamics; Prof. Subhasis Chaudhari at EE, IIT Bombay works in Computer Vision; Prof. Manoj Gopalakrishnan at EE, IIT Bombay works in biological computing; Prof. Amit Sethi in EE, IIT Bombay works in Medical Computer Vision. You could take these projects up via RnD Projects or BTech Projects in your own department.
Reading Groups, Talks - There are a plethora of events happening daily in IIT Bombay, and talks on Computer Science are no exception. The CSE Department has a number of weekly talks at the FC Kohli Auditorium, often by our own faculty (Faculty Unplugged Seminar Series / FUSS) or professors from outside IIT Bombay (CSE Department Talks). A number of reading groups are organized on a weekly / bi-weekly schedule, notably the AIML reading group. The Web and Coding Club at IIT Bombay organize informal talks on undegraduate research experiences (Reflections). Finally, the CSE Cybersecurity Club and BioBytes Student Reading Group are two student-run clubs organizing biweekly talks on computer security and biological computing respectively.

Internship Opportunities

Google Summer of Code (GSoC) / Outreachy - These are exciting opportunities in the world of open source - a chance to get to work on a real-world codebase with excellent developers from across the world. Selection into this program is completely based on prior open source experience and a final project proposal. The Web and Coding Club at IIT Bombay runs an incubation cell for GSoC projects and has a detailed guide here.
Placement Cell - A number of start-ups, companies and universities offer Computer Science based interships via the Placement Cell at IIT Bombay. Selections for the company internships are generally done via a computerized test followed by an interview. It’s a good idea to start building your competitive programming skillset for these tests. Here is a beginner’s guide.
University internship selections often use CPI, prior work, recommendation letters or an interview for selection. It’s preferred to apply to dedicated undergraduate internship programs for greater flexibility in the project. MITACS, Viterbi, SN Bose, DAAD are popular internship programs among IIT Bombay undergraduates.
Direct Applications - While it’s always better to apply to dedicated internship programs, you could directly email suitable university professors or company recruiters, seeking an internship. There is a very specific way to do this, do not spam. The best strategy is to first work with a professor in IIT Bombay and gain some research experience in some area of Computer Science. Then, contact professors who know your IIT Bombay advisor or whose work is closely aligned to your advisor’s. If you have built a good rapport with your IIT Bombay advisor, feel free to take his / her advice about this.
Seasons of Code / ITSP - The Seasons of Code is a student-run initiative where you get a chance to work with a senior undergraduate from IIT Bombay on his / her Computer Science project. This is a great chance (especially in your first two years) to get your hands dirty and find a good senior mentor (who could potentially guide you in your future career endeavours).
The Institute Technical Summer Project (ITSP) is another student-run initiative where freshers get to work on a project of their choice along with a small budget over their first year’s summer. Students are free to work on technical projects in Computer Science, Electronics, Aeromodelling, Astronomy etc and are assigned a suitable mentor.

Other Opportunities

Tech Teams - Most Tech Teams at IIT Bombay have a Software Subsystem requiring CSE enthusiasts. Have a look at the “Tech Teams” tab in the Institute Technical Council’s website.
Online Resources - A number of online courses, blogs, video tutorials can be quite useful while learning Computer Science. Here is an excellent topicwise compilation of awesome university courses to learn CS. Notably, MIT OCW and Coursera are good places to find video lectures of complete courses. Stanford has publically released a number of machine learning courses like CS224n, CS229 and CS231n.
Finally, here is a compilation of a number of topic-wise introductory programming articles and guides by The Web and Coding Club, IIT Bombay.
Competitions -
- ACM ICPC - This is a popular contest in which a number of CS students from IIT Bombay take part (and do well!). “Competitive Coding” in its true sense, preparation for this contest requires significant comfort with data structures and a lot of practice. Here is a guide to get you started. Here is a useful certification by CodeChef.
- Capture the Flag Events - These are computer security competitions which are quite popular among CS undergraduates at IIT Bombay. DEFCON CTF is one popular international level CTF. Contact the CSE Cybersecurity Club for more details.
- Hackathons - A number of hackathons are organized throughout the year, sponsored by tech companies and startups. These are generally co-organized with an institute body like The Web and Coding Club at IIT Bombay or the Computer Science Department at IIT Bombay. Notably, Microsoft organizes a two-stage hackathon yearly, named code.fun.do.

Feel free to contact me in case you would like to add something to the article!

Grad School Resources

Tue, 29 May 2018 18:30:55 +0000

Previously, I wrote an article on GRE and TOEFL preparation. This is my second blogpost on graduate school admissions. I have tried to compile all the online resources I used to prepare my graduate school application.
Most of this advice is written by professors or senior PhD students, and is focused towards PhD applicants. Several webpages shared in this blog contain a wide-spectrum of advice articles, many of which are relevant to undergraduates and graduate school applicants.

Students’ Perspective on Grad School Applications - Most recently, we have surveyed twelve recent NLP PhD applicants on their experience while applying to graduate school. Unlike the remainder of this article, this article was written by young PhD students who applied to graduate school in the last 3 years. Check it out here!

PhD Application Guides

Mor Harchol Balter’s Guide - An excellent but slightly out-dated guide on writing good PhD applications by a CMU professor. This is a very comprehensive article which starts with the pros and cons of a PhD degree, moves to the application process and ends with some advice on choosing the correct graduate program.
Philip Guo’s Five-Minute Guide - A short guide describing the initial screening process during PhD admissions. Lots of interesting videos / articles in the “related pages” section at the end of the guide. You might find the Statements of Purpose linked in this article useful. Philip Guo also has a Vlog on finding computer science PhD programs to apply to using CSRankings.
Philip Guo’s Older Guide - A more comprehensive article listing the best strategies to maximize your chances of selection. This was written while Philip Guo was still a graduate student.
Himabindu Lakkaraju and Aditya Grover’s Panel on ML PhD applications in USA, conducted in November 2020.The panel includes several faculty including Tatsu Hashimoto (Stanford), Rada Mihalcea (UMichigan), Devi Parikh (Georgia Tech), Sameer Singh (UC Irvine), and James Zou (Stanford).
Jean Yang’s Guide - A seven-part comprehensive article on the whole PhD admission process. Jean Yang’s Statement of Purpose is very well-written, and helped me considerably. This guide also contains details about what to look for during visit days.
Jason Eisner’s Guide for JHU / CLSP - A very nice article on graduate admission process, with a focus on JHU admissions into the Center for Language and Speech Processing (CLSP) (a top research group for prospective NLP / Speech students). This article contains an important section on school rankings (and why they should be taken with a grain of salt). Prof. Jason Eisner has written several excellent advice articles here, some of which are relevant to graduate school admissions. He is also quite active on Quora.

CLSP has also written a broader PhD admission FAQ here.
CMU LTI Admission Guide - This Quora answer by Scott Fahlman is very specific to to MLT / PhD admissions at LTI (Language Technologies Institute) at Carnegie Mellon University (CMU). There is another answer by Alex Rudnicky here. Scott Fahlman has written several interesting Quora answers on NLP and academica-related topics here.

Finally, here is an article by Danish Pruthi discussing the conversion statistics from an MLT degree to a PhD degree, an important question many NLP PhD candidates have.
Matthew Might’s Guide - An excellent blog focussing on the importance of good recommendation letters and prior research experience (preferrably publications). Has some nice examples of accepted and rejected applications towards the end. Matthew Might has written many comprehensive advice articles (see “Related posts”) on sending emails, doing research etc. One celebrated article (rather, a collection of pictures) is An Illustrated Guide to A PhD. He also has an article on asking for a recommendation letter.
David Evan’s Guide for Writing Emails - A popular article on the optimal strategy to emailing prospective PhD advisors. A must-read if you are planning to send out emails to professors.
Micheal Ernst’s Guide - You can see all the advice articles written by Micheal Ernst here.
Justine Sherry’s Guide - A University of Washington CSE student’s guide on graduate school applications. A bit specific to UW.
Christopher Fletcher’s Guide on Personal Statements - This guide includes his personal statements as well. He has shared some more advice articles here.
Noah Smith’s Advice (Internet Archive link) - A post-selection article summing up the factors one should consider while choosing between graduate schools. This article advises against using college rankings and focusing on the prospective advisors instead. Noah Smith has also written some advice for undergraduates considering graduate school here. Noah Smith also has a newer article (November 2020), an FAQ for PhD applications.
Tim Dettmers’ Guide on Successful PhD applications (Internet Archive link) - A new guide by a successful 2017-18 PhD applicant in natural language processing. Has some interesting statistics on the track record of people selected at top PhD programs.
Tim Dettmers’ Guide on choosing a graduate school - An excellent guide on the thought process involved while choosing a graduate school after receiving admits. Tim Dettmers also wrote an older post on this topic.
Vijay Chidambaram’s Tweet - A detailed tweet on best practices while writing SOPs. He also has an article on choosing the right graduate school.
Mark Corner’s Admission Notes on requirements and common mistakes made by international students applying for PhD programs in USA.
Andreas Madsen’s Experience on becoming an independent researcher. While this is not a strict application guide, it talks about the competitive nature of today’s PhD admission process in machine learning, and the steps he took to become an independent researcher.
Nathan Schneider’s guide on writing better statements of purpose.
Alex Tamkin’s Twitter thread on different types of advisors, tradeoffs, and what prospective PhD applicants should be looking for.
Andrew Kuznetsov’s guide on “what do I ask/look for” in a PhD Advisor. A more comprehensive version of this article (with inputs from several PhD students at CMU) can be found here.
Gautam Kamath’s article on asking for letters of recommendation.
Suchin Gururangan’s guide on writing effective personal statements.
Shriram Krishnamurthi’s video guide on applying for PhD programs in USA.
Jeff Huang’s collection of useful data (like PhD stipends, professor lists, best paper award lists) while applying to PhD programs.
Swapneel Metha’s article on writing better statements of purpose.
Rishi Bommasani’s twitter thread on applying for CS PhD programs.
Jia-Bin Huang’s tweet on applying for research opportunities

MS Application Guides

Karthik Raghunathan’s Guide - An article describing the admission process in Stanford’s MS in Computer Science program. This article after his own experience reviewing applications in the Stanford admission committee. It gives a nice idea about what application reviewers are looking for.
Shriram Krishnamurthi’s article on financial aid in computer science master’s programs.
Danish Pruthi and Mansi Gupta’s article on why university ranking is not as important as other factors (like match with advisor / lab) while choosing between graduate programs.

PhD Experiences

Philip Guo’s Experience (PhD Grind) - A first hand account of Phillip Guo’s experience completing a PhD in Computer Science. This article describes (in utmost detail) the highs and lows during a PhD in Computer Science, and ends with a few reasons describing why a PhD might be worth it. A must-read if you are confused between an MS and a PhD.
Jean Yang’s Experience

PhD Advice

These articles are guides for having a more successful PhD (after joining a PhD program).

Mark Dredze and Hannah Wallach’s Guide - A guide with a number of tips to have a successful PhD in NLP / ML. It is quite comprehensive and contains very good advice written by two top NLP researchers.
Kevin Gimpel’s Advice - A gem of an article with a unique perspective on the PhD experience. Titled “How to be a Happier PhD Student”, it’s a motivating read to recover from bad, unproductive days.
Andrej Karpathy’s Guide (A Survival Guide to a PhD) - A famous blog by an ex-Stanford PhD student. Andrej Karpathy’s articles are quite famous in the machine learning community.
H.T. Kung’s Advice
Phillip Torr’s Guides
Phil Agre’s Guide on professional skills for PhD students
Karl Stratos’ article on being a successful PhD student
Jia-bin Huang’s articles — An excellent collection of advice articles of various aspects of PhD life (making steady progress, working with mentors, cold emailing, time management, paper writing etc). Originally appeared as a series of Twitter threads.
Vijay Chidambaram’s twitter thread on choosing industry vs academia after graduation.
Maxwell Forbes’ article on why every PhD experience is different.
Ofir Press’ article on tips for junior researchers.

Research Advice

These articles are more general pieces of advice to have a more impactful research career.

David Patterson’s slides on how to have a bad career in research. Here is a newer video with the talk.
Emery Berger’s article on having real-world impact via your research.
Richard Hamming’s talk transcript on how to do world-class research. Here is a video for the talk.
John Schulman’s guide to ML Research
Michael Nielsen’s article on Effective Research
Jordan Boyd-Graber’s advice on style conventions while writing articles and preparing for conference presentations. Also has some useful tips on compiling a thesis document.
Arka Sadhu’s compilation on research advice articles: a compilation of research advice articles. While there are some overlaps between this document and Arka’s list, this document focuses on graduate school admission whereas Arka’s compilation focuses on the actual process of doing research.
Shaily Bhatt’s compilation on research advice articles
Derek Hoiem’s article on maximizing the impact of your research.
Rachit Nigam’s article on learning to fail in research. It’s an accurate description (through a memoir) of what research is like, and why failure, optimism, collaboration and executing / quickly prototyping research ideas is important. He has also written an interesting memoir on the first two years of his PhD.
Devi Parikh’s Humans of AI interviews with 16 top AI researchers on how they decide what research problems to work on.
Alex Tamkin’s advice for new researchers and working effectively with mentors.
Rowan Zeller’s blog on his job search and choosing between academia and industry.
Jason Wei’s blog on conducting effective research in AI / NLP.

GRE and TOEFL Preparation - 2

Thu, 07 Dec 2017 19:30:55 +0000

This blog is a continuation of my initial GRE / TOEFL blog and focusses on TOEFL.

TOEFL - Test of English as a Foreign Language

TOEFL (specifically iBT - Internet Based Test) is scored out of 120 marks, 30 marks for each section (Reading, Writing, Speaking and Listening). For nearly all programs under the sun, a total score of 110 is considered “safe”. For most programs, the minimum requirements hover between 90-100. However, this “minimum requirement” has a catch, since most programs ask for minimum scores in each section. A few programs with extensive Teach Assistant requirements have high Speaking score requirements (26-28). However, like the GRE, TOEFL scores are used more like a “threshold”. Your actual score is of little importance as long as your cross the threshold.

TOEFL, unlike GRE, is evaluated quite leniently. You DO NOT need all correct answers to score full marks in any section. The evaluators of the Writing and Speaking section are quite generous, and you don’t need perfect essays or speech for perfect marks.

Choosing a Date

I gave my TOEFL exam on 24th September, 2017 (for reference, my graduate applications were due in December 2017), and registered for it on 25th July, 2017. Similar to GRE, June to October are rush-hours, and it’s often hard to get weekend slots. Try to book 3-4 months in advanced. Payment require credit card (keep one handy, it is needed for graduate applications too), and the cost is a whopping 180$!

TOEFL Preparation

I studied about one week for TOEFL. This should be sufficient if you are from an English-medium school or college. Since I had given GRE a few weeks before TOEFL, I didn’t particularly prepare for the Reading and Writing sections. The reading and writing sections are similar to GRE’s reading comprehensions and AWA section, just a LOT simpler. If you were having trouble finishing the verbal or AWA sections for GRE (or haven’t given GRE yet), I suggest practising each of these sections (with a time limit) atleast 2-3 times. In my opinion, the stipulated time is just-enough to complete these sections. I will focus more on the Listening and Speaking section in this guide.

Preparation Material

I primarily used Magoosh’s YouTube videos. This video is a good starting point. MagooshTOEFL’s YouTube channel has a plethora of good videos, and post a “TOEFL Tuesday” video each week. I personally went through several videos on Listening and Speaking skills on this channel. Additionally, lingunamaria has a good TOEFL playlist, along with her personal TOEFL experience.

For practice, I used the Official TOEFL iBT Volume 1 & 2. More specifically, I used the computer-delivered versions of the tests using the provided DVD. This package contains ten real iBT TOEFL practice tests.

For Linux users, I could not succeed in installing this software using WineHQ. I eventually used a virtual machine installation of Windows 7 (using VirtualBox). This installation worked quite well for me, without any audio issue.

Diagnostic Test

Doing a diagnostic test is harder for TOEFL, since your essay writing and speaking sections will not be evaluated. I did the Listening and Speaking sections of one practice test (among the ten linked above) as diagnostic. I recorded my Speaking section’s answers, and evaluated them subjectively against some official metrics.

Listening Preparation

Listening sections are hard since you have to pick out the most important information from 5 minutes of dense audio clips. The audio is played just once, so note-making skills are critical here. It is quite hard to note down everything and difficult to understand what is going to become essential information while answering the questions.

Luckily, the questions are quite straightforward. A couple of questions replay small segments of the audio clip. After a few practice tests, I noticed that a similar pattern of questions are asked test after test. The audio clips are scripted, and the most important information is repeated 2-3 times in the clip.

Generally, student conversations are easier to retain and shorter in length. The classroom lectures are longer, and difficult to follow if you misunderstand the first few concepts. This section requires the most concentration, leave no stone unturned while preparing for this section. I personally solved most of the listening sections in the ten TOEFL practice tests (linked above).

Speaking Preparation

I was particularly nervous about the Speaking Section, since I needed a score of 26 for one of my graduate programs. I started off with this Magoosh guide, and watched a few videos on MagooshTOEFL’s YouTube channel.

I was particularly nervous about the preparation time given for each question in the Speaking section. However, you should add 10-15 seconds to the mentioned preparation time, since the question is printed on the screen while it is being read out slowly. I found it useful to write down the key ideas I intend to talk about. Occasional stammering is all right, but make sure you keep talking till the very end. It’s also important to speak slowly and clearly.

Most importantly, get over the fear / awkwardness of talking into the microphone. Most of us are not seasoned YouTubers or radio jockeys. This will only come with practice. I personally did all ten speaking practice tests (linked above) before my final TOEFL exam.

Exam Day Preparation

IMPORTANT - Make sure you have decided a list of 4 colleges to send your scores to. Do check whether the program require TOEFL scores. Some programs waiver TOEFL for Indian students, since English is one of India’s two native languages. The list of colleges need to be updated on the TOEFL portal in the night before the exam at the very latest.

Make sure you carry your passport and snacks for the 10 minute break. Be sure you completely understand the exam pattern. Water is NOT ALLOWED in the exam room. The 10 minute break is the only time you will get to use the restroom, eat, or drink something. Carrying a jacket is ALLOWED, and a good idea since the exam hall is cold. The actual TOEFL exam has an extra experimental section, and takes about four and a half hours in total.

It might get quite awkward during the exam to speak out loudly in the microphone with so many other people in the room. It’s even more awkward when you are the ONLY person in the hall currently on the Speaking section. I tried my best to return from the ten minute break at a point where a few people had begun their Speaking section, while a few people were yet to return from their break. Unfortunately, the headphones in Mumbai’s centre do not cancel background noise well and you can clearly hear other people deliver their voice essays.

My TOEFL marks were a little unexpected, but quite satisfactory. I scored 115/120, with 29 in Listening, 29 in Writing, 30 in Reading and 27 in Speaking. Most people I know did quite well in TOEFL, without significant preparation.

GRE - Graduate Record Examination

Please check the previous guide.