How to Repeatedly Append to a String in Python

See Python: Tips and Tricks for similar articles.

In Python, if you need to repeatedly append to a string, you should convert it to a list, append your items to that list, and then join the list back into a string after you've made all the additions.

To illustrate, imagine you are reading through a long log file looking for lines that begin with "ERROR:" You want to store all those lines in a string and write them out to a new file.

Your first inclination might be to do that like this:

with open('log.txt') as f:
    log = f.readlines()

new_log = ''
for line in log:
    if line[:6] == 'ERROR:':
        new_log += line
        
with open('newlog.txt','w') as f:
    f.write(new_log)

But as Python strings are immutable, that method requires creating new memory allocation each time you modify the string.

Instead, you should do the following:

  1. Open and read the log file into a list (e.g., log):
    with open('log.txt') as f:
        log = f.readlines()
  2. Create a new empty list (e.g., new_log):
    new_log = []
  3. Loop through the log list looking for matches and append each matching line to the new_log list:
    for line in log:
        if line[:6] == 'ERROR:':
            new_log.append(line)
  4. Join new_log on an empty string and write it to the new log file:
    with open('newlog.txt','w') as f:
        f.write(''.join(new_log))

If you're only making a few changes to the string, you won't notice much difference, but if you're making thousands, this method is much more efficient.


Related Articles

  1. Fixing WebVTT Times with Python
  2. Using Python to Convert Images to WEBP
  3. Scientific Notation in Python
  4. Understanding Python’s __main__ variable
  5. Converting Leading Tabs to Spaces with Python
  6. Python Clocks Explained
  7. Python: isdigit() vs. isdecimal()
  8. Maximum recursion depth exceeded while calling a Python object
  9. When to use Static Methods in Python? Never
  10. Finally, a use case for finally – Python Exception Handling
  11. Python Coding Challenge: Two People with the Same Birthday
  12. How to Create a Simple Simulation in Python – Numeric Data
  13. Collatz Conjecture in Python
  14. Associate Python Files with IDLE
  15. How to find all your Python installations on Windows (and Mac)
  16. Python Color Constants Module
  17. A Python Model for Ping Pong Matches
  18. Bulk Convert Python files to IPython Notebook Files (py to ipynb conversion)
  19. pow(x, y, z) more efficient than x**y % z and other options
  20. Python’s date.strftime() slower than str(), split, unpack, and concatenate?
  21. Bi-directional Dictionary in Python
  22. Creating an Email Decorator with Python and AWS
  23. Change Default autosave Interval in JupyterLab
  24. Basic Python Programming Exercise: A Penny Doubled Every Day
  25. Simple Python Script for Extracting Text from an SRT File
  26. Python Virtual Environments with venv
  27. Mapping python to Python 3 on Your Mac
  28. How to Make IDLE the Default Editor for Python Files on Windows
  29. How to Do Ternary Operator Assignment in Python
  30. How to Convert Seconds to Years with Python
  31. How to Create a Python Package
  32. How to Read a File with Python
  33. How to Check the Operating System with Python
  34. How to Use enumerate() to Print a Numbered List in Python
  35. How to Repeatedly Append to a String in Python (this article)
  36. Checking your Sitemap for Broken Links with Python
  37. How to do Simultaneous Assignment in Python
  38. Visual Studio Code - Opening Files with Python open()
  39. How to Slice Strings in Python
  40. How Python Finds Imported Modules
  41. How to Merge Dictionaries in Python
  42. How to Index Strings in Python
  43. How to Create a Tuple in Python