How to Repeatedly Append to a String in Python

See Python: Tips and Tricks for similar articles.

In Python, if you need to repeatedly append to a string, you should convert it to a list, append your items to that list, and then join the list back into a string after you've made all the additions.

To illustrate, imagine you are reading through a long log file looking for lines that begin with "ERROR:" You want to store all those lines in a string and write them out to a new file.

Your first inclination might be to do that like this:

with open('log.txt') as f:
    log = f.readlines()

new_log = ''
for line in log:
    if line[:6] == 'ERROR:':
        new_log += line
        
with open('newlog.txt','w') as f:
    f.write(new_log)

But as Python strings are immutable, that method requires creating new memory allocation each time you modify the string.

Instead, you should do the following:

Open and read the log file into a list (e.g., log):

with open('log.txt') as f:
    log = f.readlines()

Create a new empty list (e.g., new_log):
```
new_log = []
```
Loop through the log list looking for matches and append each matching line to the new_log list:
```
for line in log:
    if line[:6] == 'ERROR:':
        new_log.append(line)
```

Join new_log on an empty string and write it to the new log file:

with open('newlog.txt','w') as f:
    f.write(''.join(new_log))

If you're only making a few changes to the string, you won't notice much difference, but if you're making thousands, this method is much more efficient.