020 Loops, conditions, and dictionaries

Use a loop to clean the punctuation off every word of the rhyme, choose what code runs with conditions, count every word with a dictionary, and write your first function.

Introduction

Last week words.count("twinkle") said zero, even though “twinkle” is in the rhyme four times. Every “twinkle” in our list still had a comma stuck to it. To fix that, we need to do something to every word in the list, which is what a loop is for. In this tutorial you will write loops, choose what code runs with if and else, count words with a dictionary, and package your work into a function you can reuse. In the next tutorial you will use all of them to build the counting model from class in Python.

Skim the sections on loops and if statements, but type the counting loop and the clean function as they are written here, because the next tutorial builds on both. Do the Your turn questions, and then the ones under Going further.

Get this week’s lab

This week you will get your lab from GitHub. Follow Getting and handing in work with GitHub to join the course’s organization, accept this week’s assignment from its link on the schedule, clone the repository into your CSCI295 folder, open it in VS Code, and run uv sync. The repository has the rhyme in data/raw/the_star.txt and a much longer text, alice.txt, that we will use in the next tutorial.

Make a notebook called repeating_and_deciding.ipynb in the notebooks folder, and choose the .venv kernel. Start the notebook with a Markdown cell with a title, and then read in the rhyme the way you did last week.

from pathlib import Path

DATA_RAW = Path("../data/raw")

text = (DATA_RAW / "the_star.txt").read_text(encoding="utf-8")
words = text.lower().split()
words[0:6]
['twinkle,', 'twinkle,', 'little', 'star,', 'how', 'i']

Save the notebook, and make your first commit and push it, as described in the GitHub guide’s Committing and pushing. Check on GitHub that it arrived, and then carry on here.

Loops

A for loop runs the same code once for each item in a list. Here is a loop that prints the first six words, one per line.

for word in words[0:6]:
    print(word)
twinkle,
twinkle,
little
star,
how
i

Read it as “for each word in the first six words, print the word.” Python is strict about three details here:

  • The first line ends with a colon, :.
  • The code that runs each time through the loop (the body of the loop) is indented by four spaces. VS Code indents the next line for you when you hit return after the colon. When the body is done, press return and then delete (backspace on Windows) to remove the indent from the next line. If the indenting is missing or uneven, Python stops with an IndentationError.
  • The name after for (here word) is a variable. Each time through the loop, Python sets it to the next item in the list, so the body runs once with word equal to 'twinkle,', then again with 'twinkle,', then with 'little', and so on.

The loop ends where the indenting ends. Compare these two cells, and predict how many lines each one prints before you run them.

for word in words[0:3]:
    print(word)
    print("---")
twinkle,
---
twinkle,
---
little
---
for word in words[0:3]:
    print(word)
print("---")
twinkle,
twinkle,
little
---

In the second cell, the last line is not indented, so it is not part of the loop. It runs once, after the loop is done.

Cleaning every word

The string method strip removes characters from the beginning and end of a string. We tell it which characters to remove, here the punctuation marks that appear in the rhyme.

"twinkle,".strip(".,!?;:")
'twinkle'

To clean every word, we start with an empty list and use a loop to add each cleaned word to it with the list method append.

clean_words = []
for word in words:
    clean_words.append(word.strip(".,!?;:"))

clean_words[0:6]
['twinkle', 'twinkle', 'little', 'star', 'how', 'i']

Now let’s count again.

clean_words.count("twinkle")
4

The count is now four. Notice that we did not change the original list. words still has the punctuation, because we made a new list, clean_words. We kept the original for the same reason we never edit the files in data/raw.

If statements

Often we want the code to do different things in different situations. An if statement runs its indented code only when a condition is True. Conditions are usually comparisons like the ones you met last week, such as 2 + 2 == 4, and they are either True or False.

for word in clean_words[0:8]:
    if word == "twinkle":
        print(word, "<-- found one")
    else:
        print(word)
twinkle <-- found one
twinkle <-- found one
little
star
how
i
wonder
what

The else part runs when the condition is False, and you can leave it out. Like for, both if and else end with a colon and have their own indented code. Here the if statement is inside the loop, so its code is indented twice.

Here are the comparisons you will use most often:

Code Means
a == b a equals b
a != b a does not equal b
a < b, a > b less than, greater than
a <= b, a >= b less than or equal to, greater than or equal to
a in b a is an item in the list b, a piece of the string b, or a key of the dictionary b

For example, which words in the rhyme are longer than five letters?

for word in clean_words:
    if len(word) > 5:
        print(word)
twinkle
twinkle
little
wonder
diamond
blazing
nothing
shines
little
twinkle
twinkle

And we can ask whether a word appears in the rhyme at all:

"star" in clean_words
True
"moon" in clean_words
False

Counting every word with a dictionary

The count method answers the question for one word at a time. To count every word, we need somewhere to keep a running total for each one, so we will use a dictionary. It stores keys and a value for each key, like a word and its count. Dictionaries are written with curly braces, with a colon between each key and its value:

example = {"twinkle": 4, "star": 1}
example["twinkle"]
4

Square brackets look up the value for a key, and you can use them to add a key or change its value:

example["night"] = 1
example
{'twinkle': 4, 'star': 1, 'night': 1}

Now we can count every word in the rhyme, starting with an empty dictionary. For each word, if it is already a key, we add one to its count. If not, it is the first time we have seen the word, so its count starts at one.

counts = {}
for word in clean_words:
    if word in counts:
        counts[word] = counts[word] + 1
    else:
        counts[word] = 1

counts
{'twinkle': 4,
 'little': 2,
 'star': 1,
 'how': 1,
 'i': 1,
 'wonder': 1,
 'what': 1,
 'you': 2,
 'are': 1,
 'up': 1,
 'above': 1,
 'the': 4,
 'world': 1,
 'so': 1,
 'high': 1,
 'like': 1,
 'a': 1,
 'diamond': 1,
 'in': 1,
 'sky': 1,
 'when': 2,
 'blazing': 1,
 'sun': 1,
 'is': 1,
 'gone': 1,
 'he': 1,
 'nothing': 1,
 'shines': 1,
 'upon': 1,
 'then': 1,
 'show': 1,
 'your': 1,
 'light': 1,
 'all': 1,
 'night': 1}

Take a minute to trace through the loop for the first few words by hand. What does counts look like after the first word? After the second? After the third? (A dictionary keeps its keys in the order they were first added, which is why the words come out in the order they first appear in the rhyme.)

The dictionary has one key for each different word:

len(counts)
35

So the rhyme has 44 words but only 35 different ones. And now we can look up any word:

counts["twinkle"]
4

What happens if you look up a word that is not a key, like counts["moon"]? Try it, and read the error message. (Then turn that line into a comment, so it does not stop Run All.)

Writing a function

We have cleaned the words of one text, and in the next tutorial we will want to do the same to a much longer one. Rather than copy the code, we can turn it into our own function. You have been using functions all along (len, print, and type), and now you will write one.

def clean(text):
    """Split text into lower-case words and strip punctuation from each one."""
    words = []
    for word in text.lower().split():
        word = word.strip('.,!?;:"()')
        if word != "":
            words.append(word)
    return words

Nothing happens when you run this cell, because it only defines the function and does not run it. Here is what each part does:

  • def clean(text): says we are defining a function named clean that takes one input, which will be called text inside the function.
  • The line in triple quotes is a docstring, a short description of what the function does. It is what help(clean) shows.
  • The indented lines are the body of the function. This body cleans more kinds of punctuation than before (quotation marks and parentheses too), and it skips any “word” that turns out to be empty after stripping. An exclamation point standing by itself, for example, would strip down to an empty string. The characters to strip are in single quotes this time, because one of them is a double quotation mark.
  • return words sends the finished list back to whoever called the function.

Now we can call the function on the rhyme, or on any text at all:

clean(text)[0:6]
['twinkle', 'twinkle', 'little', 'star', 'how', 'i']
clean('"Curiouser and curiouser!" cried Alice.')
['curiouser', 'and', 'curiouser', 'cried', 'alice']

The variables inside a function belong to the function. The words inside clean is a separate variable from the words list we made at the top of the notebook, which has not changed:

words[0]
'twinkle,'

Your turn

Work below the tutorial code, with a Markdown cell before each question.

  1. Loop over clean_words and print each word that starts with the letter “t”. (The string method startswith will help. Use help(str.startswith).) Then make your code count them, repeats included, instead of counting by hand.
  2. Use the counts dictionary and a loop to find the word that appears most often in the rhyme, and how many times it appears. Before the loop, set best_word = "" and best_count = 0. Inside the loop, when a word’s count is larger than best_count, update both. (Looping over a dictionary gives you its keys, as in for word in counts:.) Is there a tie? What does your loop do when there is one?
  3. What-if questions. Answer in Markdown, and check your answers with code when you can.
    1. In the cleaning loop, what would happen if clean_words = [] were inside the loop (indented) instead of before it?
    2. Why did we skip empty strings in the clean function? Write a string for which the clean function would give a different result if we did not.
    3. Our clean function turns “Star” and “star” into the same word. Can you think of a text where that would be a mistake?
  4. In a Markdown cell, explain in your own words the difference between a list and a dictionary, and give an example (from any field you like) of data that fits naturally into each.

These are optional.

  • Write a function called count_words that takes a list of words and returns a dictionary of counts. Check that count_words(clean_words) == counts is True.
  • Python programmers often write the counting step as counts[word] = counts.get(word, 0) + 1. Look up the dictionary method get and explain why this line does the same job as our if and else.
  • Is it faster to check whether a word is in a list or in a dictionary? Why might it be?

What to hand in

When you have finished, click Restart and then Run All. Check that everything runs and that the results still match the tutorial and your notes, and save. Then commit and push your work as described in the GitHub guide, so that you have a backup. You will keep working on this same lab in the next tutorial, and both of this week’s notebooks are handed in together by the week 2 deadline on the schedule.

Back to top