1. File read and write operations

Everything in the Linux system is a file, so we inevitably have to deal with files, and we often read and write files. E.g:

cat /etc/password  # read file
vim /etc/password  # read and write files
echo test > /tmp/abc.txt  # overwrite file
echo text >> /tmp/abc.txt  # append to file

In the above content, we all read and write text files, and there are also read and write operations to binary files in the computer. How to implement it in Python?

1. File opening method - open built-in function

Whether reading a file or writing a file, our first step is to open the file.

As the "key" to open the door of a file, the built-in function open() provides a general interface for initializing input/output (I/O) operations. When a file is successfully opened, it will return a file object, otherwise an error will be raised.

(1) Basic grammar

[External link image transfer failed, the source site may have anti-leech mechanism, it is recommended to save the image and upload it directly (img-FhPn4usm-1653320533809)(day03.assets/image-20210810162704041.png)]

To use a file in any way—even just to print its contents—the file must be opened so that it can be accessed.

(2) Parameter introduction

**file_name:** indicates the path of the file we want to open

**mode:** how to open the file

file mode	operate
r	Open for reading (error if the file does not exist)
w	Open for writing (empty if the file exists, create if it does not)
a	open in append mode
b	Open in binary mode

**file_object: **File operation object, all subsequent read and write operations on the file need to pass through this object, instead of directly operating the data in the file.

2. File read operation

To use the information in a text file, you first need to read the information into memory. To do this, we can either read the entire contents of the file at once, or step-by-step, one line at a time.

(1) read method - read file

• The first parameter of the open function is the name of the file to open (the file name is case sensitive)
  • If the file exists, return the file operation object
  • If the file does not exist, an exception will be thrown
• The read method can read and return all the contents of the file at once
• The close method is responsible for closing the file
  • If you forget to close the file, it will consume system resources and affect subsequent access to the file
• Note: After the read method is executed, it will move the file pointer to the end of the file

# 1. Open - file name needs to pay attention to case
file = open("file path", "r")

# 2. Read
text = file.read()  # Read all the contents of the file at one time
print(text)  # Print the read content to the console

# 3. Close
file.close()

(2) File pointer

• file pointer mark where to start reading data
• When a file is first opened, usually the file pointer points to the beginning of the file
• When the read method is executed, the file pointer will move to the end of the read content
  • Moves to end of file by default
• When the file is reopened, the file pointer re-points to the beginning of the file

legend
think

• If the read method is executed once and all the content is read, can the content be obtained by calling the read method again?

Answer

• No, because after the first read, the file pointer moves to the end of the file, and calling it again will not read anything

(3) readline method - read line by line

• By default, the read method will read all the contents of the file into memory at once

• If the file is too large, the memory usage will be very serious

• The readline method can read one line at a time

• After the method is executed, it will move the file pointer to the next line, ready to read again

Case: Correct posture for reading large files

# open a file
file = open("file path", mode="r")

while True:
    # read a line
    text = file.readline()

    # Determine whether to read the content
    if len(text) == 0:
        break

    # There is already a \n at the end of each line read
    print(text, end="")

# close file
file.close()

(4) readlines method

The readlines() method reads all (remaining) lines and returns them as a list of strings

legend

• If you need to process the file line by line, you can iterate the file with a for loop
• Iterating over files is similar to processing other sequence types of data

f = open("/tmp/passwd", mode="r")
for line in f:  # Equivalent to for line in f.readlines():
    print(line, end=" ")

3. File write operation

(1) write method - write file

• The write() built-in method is the opposite of read() and readline()
  • It writes strings containing text data or binary data blocks to a file
• When writing a file, the end-of-line mark is not automatically added, requiring manual input from the programmer, and the number of bytes written is returned
  

(2) writelines method

• Like readlines(), the writelines() method operates on lists
• It takes a list of strings as arguments and writes them to a file
• Line terminators are not added automatically, so if needed, you must add line terminators to the end of each line before calling writelines()

Case : writelines

f = open("./d.txt", mode="w")
f.writelines(['1st line.\n', '2nd line.\n','3rd line.\n'])
f.close()

legend:

[External link image transfer failed, the source site may have an anti-leech mechanism, it is recommended to save the image and upload it directly (img-bQFzXxVJ-1653320533818)(day03.assets/image-20210820103515549.png)]

4. with clause

• The with statement is used to simplify the code

• Put the operation of opening the file in the with statement, the file will be closed automatically after the code block ends

• The logic of reading and writing files has not changed, it has become just the way of writing

Case: with

with open('/tmp/passwd', mode="r") as f:
    f.readline()

practise

# Simulate cp operation
# 1. Create the cp.py file
# 2. "Copy" /usr/bin/ls to /tmp directory
# 3. Do not modify the original file

version one

# Create two object variables, f1 is the original file; f2 is the target file to write data
# Because it is a binary file, it is read and written in bytes.
f1 = open('/usr/bin/ls', mode='rb')
f2 = open('/tmp/list', mode='wb')
# Read data from the original file /usr/bin/ls with f1 and store the data in the variable data
# f2 writes the contents of the variable data to the target file /tmp/list
data = f1.read()
f2.write(data)
#close file f1 and file f2
f1.close()
f2.close()

Check whether the md5 value of the original file and the target file are equal

[root@localhost xxx]# md5sum /tmp/ls /tmp/list

Version 2: Optimization

# Create two variables, src_fname stores the source file path; dst_fname stores the destination file path
src_fname = '/usr/bin/ls'
dst_fname = '/tmp/list2'
# Create two object variables, src_fobj is to open the original file; src_fobj is to open the target file
# Because it is a binary file, it is read and written in bytes.
src_fobj = open(src_fname, mode='rb')
dst_fobj = open(dst_fname, mode='wb')
while 1:  # Unsure the number of reads, use a while loop
    data = src_fobj.read(4096)     # 4k read from metafile each time
    if len(data) == 0:            # data is 0, indicating that the pointer points to the end, and the data is read
        break                      # exit the entire while loop
    else:
       dst_fobj.write(data)        # Write the data data to the list2 file

src_fobj.close()     # Close the original file /usr/bin/ls
dst_fobj.close()     # close target file /tmp/lists

2. Function

1. Quick experience

• The so-called function is to organize the code blocks with independent functions into a small module, which can be called when needed
• The use of the function consists of two steps:
  1. Defining Functions - Encapsulating Independent Functions
  2. Calling functions - enjoying the fruits of encapsulation

Case: Laundry

# laundry in the morning
print("draw water")
print("do the laundry")
print("spin dry")
# Laundry at noon
print("draw water")
print("do the laundry")
print("spin dry")
# laundry at night
print("draw water")
print("do the laundry")
print("spin dry")

Found the problem: we encapsulated code with independent functions into a function

def washing_machine():  # The washing machine can do it for us
    print("draw water")
    print("do the laundry")
    print("spin dry")
# optimized code
# laundry in the morning
washing_machine()  # Turn on the washing machine switch
# Laundry at noon
washing_machine()  # Turn on the washing machine switch
# laundry at night
washing_machine()  # Turn on the washing machine switch

2. Function creation and call

(1) Create function

Functions are created with def statements with the following syntax:

def Function name(parameter list):  # The specific situation is treated according to the specific situation, and the parameters are optional.
	"""function description docstring"""
    function encapsulated code
    ......

The header line consists of the def keyword, the name of the function, and the set of parameters (if any)

The remainder of the def clause includes an optional but strongly recommended docstring, and the required function body

The function name should be named according to the naming rules for identifiers

• Can consist of letters, underscores and numbers
• cannot start with a number
• Cannot have the same name as a keyword

def washing_machine():  # The washing machine can do it for us
    print("draw water")
    print("do the laundry")
    print("spin dry")

(2) call function

Use a pair of parentheses () to call a function, if there are no parentheses, it's just a reference to the function

Any input parameters must be placed in parentheses

legend:

Case: Add washing powder

def washing_machine():  # The washing machine can do it for us
    print("draw water")
    print("Add laundry detergent! ! !")
    print("do the laundry")
    print("spin dry")
# laundry in the morning
washing_machine()
# Laundry at noon
washing_machine()
# laundry at night
washing_machine()

Summarize

• After the function is defined, it only means that the function encapsulates a piece of code.
• If the function is not actively called, the function will not be actively executed

think

• Can the function call be placed above the function definition?

  • cannot!
  • Because before using the function name to call the function, you must ensure that Python already knows the existence of the function
  • Otherwise, the console will prompt NameError: name 'menu' is not defined (name error: the name menu is not defined)

3. Parameters of the function

(1) Formal parameters and actual parameters

• Formal parameters: When defining a function, the parameters in parentheses are used to receive parameters and are used as variables inside the function
• Actual parameters: When calling a function, the parameters in parentheses are used to pass data into the function.

question

When we want to wash other things, we have to manually change the code inside the method:

def washing_machine():  # The washing machine can do it for us
    print("draw water")
    print("Add laundry detergent! ! !")
    print("wash the sheets")  # quilt cover
    print("spin dry")

Inside the function there are certain changing values:

def washing_machine():  # The washing machine can do it for us
    print("draw water")
    print("Add laundry detergent! ! !")
    print("do the laundry")
    print("spin dry")
washing_machine()
def washing_machine():  # The washing machine can do it for us
    print("draw water")
    print("Add laundry detergent! ! !")
    print("wash the sheets")
    print("spin dry")
washing_machine()
......

Think about what's wrong

Functions can only handle fixed data

How to solve?

• It would be nice if the data to be processed could be passed inside the function when calling the function!

(2) Passing parameters

• Fill in the parameters inside the parentheses after the function name
• Use , to separate multiple parameters
• When calling a function, the number of actual parameters needs to be the same as the number of formal parameters, and the actual parameters will be passed to the formal parameters in turn

def washing_machine(something):  # The washing machine can do it for us
    print("draw water")
    print("Add laundry detergent! ! !")
    print("wash" + something)
    print("spin dry")
# do the laundry
washing_machine("clothing")
# wash the sheets
washing_machine("sheet")

legend

(3) Function

• function, which organizes code blocks with independent functions into a small module, which is called when needed
• The parameters of the function increase the versatility of the function, and can adapt to more data for the same data processing logic
  1. Inside the function, use the parameters as variables to perform the required data processing
  2. When the function is called, according to the parameter order defined by the function, the data that you want to process inside the function is passed through the parameters

Exercise 1: Define a function that computes the sum of two numbers

positional parameters

Similar to shell scripts, the program name and arguments are passed to the python program as positional arguments, which are received using the argv list of the sys module

import sys
print(sys.argv[1], sys.argv[2])  #Analog shell: $1 $2
def get_sum(a,b):
    print(a+b)
#positional parameters
#a=sys.argv[1],b=sys.argv[2]
#The parameters passed through the terminal are all string types----type conversion is required here
get_sum(int(sys.argv[1]),int(sys.argv[2]))

legend

default parameters

A default parameter is a parameter with a default value declared. Because the parameter is given a default value, it is also allowed to not pass a value to the parameter when the function is called.

def get_sum(num1, num2, num3=10):
    print(num1 + num2 + num3)
# default parameters
# If the parameter is not manually assigned, then the default value is used
# When defining a function, parameters with default values ​​must be placed after parameters without default values.
get_sum(1,2,3) # num01=1, num02=2, num03=3    6
get_sum(1,2)   # num01=1, num02=2, num03=10   13


def get_sum1(num01=5, num02):  report an error
      print(num01 + num02)


def get_sum2(num1=10, num2=20, num3=30):
    print(num1 + num2 + num3)
get_sum2()     # 60

4. The return value of the function

• In program development, sometimes it is desirable to tell the caller a result after the execution of a function, so that the caller can do follow-up processing for the specific result.
• The return value is the final result given to the caller after the function completes its work
• Use the return keyword in a function to return the result
• On the calling side of the function, you can use a variable to receive the return result of the function

Note: return means return, which means that the execution of the method ends, and the subsequent code will not be executed

Case: Calculate the sum of any two numbers

After washing the clothes, we need to use the return keyword to return the price of each order and tell the cashier:

def get_sum(num01, num02):
    return num01 + num02

legend

no return

If there is no return statement inside the method, it will return None by default, that is, return None

Exercise 3: Fibonacci sequence function

Fibonacci sequence function

• Change Fibonacci sequence code to function
• The length of the sequence is specified by the user
• require the result to be returned with return

Version 1: Direct printing inside the method

def gen_fib():
    fib = [0, 1]  # Definition list specifying the initial two values ​​of the Fibonacci sequence
    n = int(input('length: '))  # Define a variable n, which is the number of elements in the list fib that the user wants to see
    for i in range(n - 2):
        fib.append(fib[-1] + fib[-2])
    print(fib)  # print list fib
# Call the function gen_fib() twice
gen_fib()
gen_fib()

Version 2: with return value

def gen_fib():
    fib = [0, 1]
    n = int(input('length: '))
    for i in range(n - 2):
        fib.append(fib[-1] + fib[-2])
    return fib  # return the last generated list fib
# Call the function gen_fib()
print(gen_fib())

Version 3: with parameters

def gen_fib(n):
    fib = [0, 1]
    for i in range(n - 2):
        fib.append(fib[-1] + fib[-2])
    return fib  # return the last generated list fib
# Define the list nlist, the length of the Fibonacci sequence to be generated, as the element of the list nlist
nlist = [10, 8, 6]
# Use the for loop to pass the actual parameters (elements in nlist) to the function gen_fib(n) to get three sets of Fibonacci numbers 
for i in  nlist:
    print(gen_fib(i))

Exercise 4: Copy File Function

• Modify the copy program in the file exercise
  • Change the program to the form of a function
  • Source and object files require passing parameters
  • Argument requirements come from the command line

version 1:

# Define the function copy() to implement the function of specifying a single file copy
def copy():
    src_name = '/usr/bin/ls'
    dst_name = '/tmp/list3'
# Open the source file as read-only bytes and assign it to the variable src_fobj
# Open the source file for writing bytes and assign it to the variable dst_fobj
    src_fobj = open(src_name, 'rb')
    dst_fobj = open(dst_name, 'wb')
    while 1:
        data = src_fobj.read(4096)  
        if len(data) == 0:
            break
        dst_fobj.write(data)
        
    src_fobj.close()
    dst_fobj.close()    
copy()  # Call the function copy() to copy the file

Version 2:

def copy(src_name, dst_name):  # Define the function copy() to implement the copy operation of any file
# Open the source file as read-only bytes and assign it to the variable src_fobj
# Open the source file for writing bytes and assign it to the variable dst_fobj
    src_fobj = open(src_name, 'rb')
    dst_fobj = open(dst_name, 'wb')
    while 1:
        data = src_fobj.read(4096)
       if len(data) == 0:
            break
        dst_fobj.write(data)
    src_fobj.close()
    dst_fobj.close()  
copy(source file path, target file path)  # Use the sys module's, argv list to get positional arguments

3. Module basis

1. Define the module

basic concept

• A module is a form of logically organizing python code
• When the amount of code becomes quite large, it is better to divide the code into some organized code segments, provided that they interact with each other
• These code fragments are related to each other, it may be a class containing data members and methods, or it may be a set of related but independent operation functions

2. Import the module (import)

• Import modules using import
• Module properties are called via the method "modulename.properties"
• You can also import individually if you only need certain properties in the module

(1) Why do you need to import modules?

Can improve development efficiency and simplify code

legend

use correctly

# test.py, put file_copy.py in the same directory
# Requirement: To copy /etc/passwd to /tmp/passwd
src_path = "/etc/passwd"
dst_path = "/tmp/passwd"
# How to copy?
# call a method in an existing module
#    - Highly recommended, simple and rude
#    - Just use the method of file_copy.py directly

# Import method 1: import the module directly
import file_copy  # Be aware of path issues
file_copy.copy(src_path, dst_path)

# Import method 2: only import the copy method of the file_copy module
from file_copy import copy  # If importing multiple modules at the same time from file_copy import *
copy(src_path, dst_path)

# Import method 4: import module with alias as
import file_copy as fc
fc.copy(src_path, dst_path)

(2) Commonly used methods of importing modules

• One line guides into a module, you can import multiple lines, for example: import random
• Only import certain methods in the module, for example: from random import choice, randint

3. Module loading (load)

• A module is only loaded once, no matter how many times it is imported
• Loading only once prevents multiple imports when code is executed multiple times
• Prevents infinite mutual loading if two files are imported into each other
• When the module is loaded, the top-level code is automatically executed, so it is the best programming practice to only put functions in the top-level of the module

4. The return value of the function

Module Features

When a module is imported, it will first execute all the programs in the module completely.

Case: Test Content Results of Isolating Imported Modules

# foo.py
print(__name__)

# bar.py
import foo  # Importing foo.py will execute the code in foo.py once, so print(__name__) in foo will be executed

result:

# foo.py   ->   __main__   
When the module file is executed directly,__name__value of'__main__'


# bar.py   ->   foo        
When a module is imported by another file,__name__The value is the name of the module

So our standard format for executing code in a Python module later on:

def test():
    ......
if __name__ == "__main__":
    test()

Isolate the test content of a.py

Exercise: Generate random passwords

Create the randpass.py script with the following requirements:

1. Write a program that generates an 8-digit random password

2. Use random's choice function to randomly pick out characters

3. Improve the program, the user can decide how many digits to generate the password

Version one:

import random  # Call the random number module random
# Define the variable all_chs to store all choices of passwords;
# Define the variable result, which stores 8-bit random numbers, and the initial value is ''
all_chs = '1234567890abcdefghigklmnopqrstuvwxyzABCDEFGHIGKLMNOPQLMNUVWXYZ'
result = ''
#Use a for loop, loop 8 times, randomly generate a character from all_chs each time, and splicing it into result
for i in range(8):
    ch = random.choice(all_chs)
    result += ch
print(result)  # Output the results, right-click to execute [Run 'randpass'] to view the results

Version 2 (optimized): functional program, and can specify password length, operate in randpass.py file

import random  # Call the random number module random
# Define the variable all_chs to store all choices of passwords;
all_chs = '1234567890abcdefghigklmnopqrstuvwxyzABCDEFGHIGKLMNOPQLMNUVWXYZ'
def randpass(n=8):  # Use def to define the function randpass() to generate a random 8-bit password
    result = ''
    for i in range(n):
        ch = random.choice(all_chs)
        result += ch
    return result  # return to the function to return the password
if __name__ == '__main__':  # Test code block, when __name__ is called as a python file, execute the code block
    print(randpass(8))
    print(randpass(4))

Version 3: Character selection for random passwords can call the module

# Call the random number module random
# In the variables ascii_letters and digits in the string module, uppercase and lowercase letters and all numbers are defined
# [Ctrl + left mouse button] You can see the content of the module file of ascii_letters
import random
from  string import ascii_letters, digits
# Define the variable all_chs to store all choices of passwords;
all_chs = ascii_letters + digits
# Use def to define the function randpass() to generate a random 8-bit password
def randpass(n=8):
    result = ''
    for i in range(n):
        ch = random.choice(all_chs)
        result += ch
    return result  # return to the function to return the password
# Test code block, when __name__ is called as a python file, execute the code block
if __name__ == '__main__':
    print(randpass(8))
    print(randpass(4))