8.1 Middleware

As the name suggests, middleware is a processing process between request and response processing. It is relatively lightweight and changes the input and output of django globally. Because the change is the overall situation, it needs to be cautious and practical. If it is not used well, it will affect the performance.

Definition of Django's middleware:

Middleware is a framework of hooks into Django's request/response processing. 
It's a light, low-level "plugin" system for globally altering Django's input or output.

MiddleWare，yes Django request/Hook framework for response processing.
It is a lightweight, low-level "plug-in" system for global change Django Input or output of. [input refers to the client like the server django Send data, output refers to django The result of processing data according to the client's requirements is returned to the client]

If you want to modify the request, for example, it is sent to the HttpRequest object in the view. Or you want to modify the HttpResponse object returned by the view, which can be implemented through middleware.

django framework declares a lot of MIDDLEWARE, which has a variety of uses, some are not used, some are enabled by default. The MIDDLEWARE that is opened for use is in settngs Py MIDDLEWARE.

Django's default Middleware:

MIDDLEWARE = [
    'django.middleware.security.SecurityMiddleware',
    'django.contrib.sessions.middleware.SessionMiddleware',
    'django.middleware.common.CommonMiddleware',
    'django.middleware.csrf.CsrfViewMiddleware',
    'django.contrib.auth.middleware.AuthenticationMiddleware',
    'django.contrib.messages.middleware.MessageMiddleware',
    'django.middleware.clickjacking.XFrameOptionsMiddleware',
]

8.1.1. Custom Middleware

(1) Define Middleware

Create a file to store custom middleware. Here, choose to create mdws in app01 Py file:

from django.utils.deprecation import MiddlewareMixin


class Md1(MiddlewareMixin):

    def process_request(self, request):
        print("Md1 request")
        # return HttpResponse("Md1 interrupt")    # intercept

    def process_response(self, request, response):
        print("Md1 return")
        return response


class Md2(MiddlewareMixin):

    def process_request(self, request):
        print("Md2 request")
        # return HttpResponse("Md2 interrupt")

    def process_response(self, request, response):
        print("Md2 return")
        return response

1. process_request returns None by default. If it returns None, the next middleware process will continue_ request； Once the response body object is returned, the return is intercepted.
2. process_response must have a formal parameter response and return response; This is the response body returned by the view function, which is passed on to the final client like a baton.

(2) Register Middleware

MIDDLEWARE = [
    ...
    'app01.mdws.Md1',
    'app01.mdws.Md2'
]

(3) Build index route

# path('index/', views.index),


def index(request):
    print("index View function execution...")
    return HttpResponse("hello yuan")

Spooling results:

Md1 request
Md2 request
index View function execution...
Md2 return
Md1 return

Therefore, from the results, we can see the execution sequence of middleware:

8.1.3. Middleware application

1. Restrict IP access frequency

The frequency of some IP access servers is too high to intercept. For example, the limit cannot exceed 20 times per minute.

8.2 cookies and Session

We know that HTTP protocol is stateless, that is to say, each request is independent! Unable to record the status of the previous request. But cookies can be used in HTTP protocol to complete session tracking! In Web development, session is used to complete session tracking, and the underlying layer of session depends on Cookie technology.

8.2.1,cookie

Cookie is translated into Chinese as "little dessert", which means "little cookie". In HTTP, it represents a small dessert sent by the server to the client browser. In fact, cookies are key value structures, similar to a dictionary in python. Send the response from the server to the client browser. Then the client browser will save the cookie and send it to the server the next time it visits the server. A cookie is a key value pair created by the server and sent to the client through a response. The client will save the cookie and mark the source of the cookie (which server's cookie). When the client sends a request to the server, all the server cookies will be included in the request and sent to the server, so that the server can recognize the client!

Cookies can be understood as local storage files of key value structure created by each browser for each server

(1) cookie flow chart

(2) cookie syntax

# (1) Set cookie s:
res = HttpResponse(...) or rep ＝ render(request, ...) or rep ＝ redirect() 
res.set_cookie(key,value,max_age...)
res.set_signed_cookie(key,value,salt='Encrypting salt',...)　
# (2) Get cookie s:
request.COOKIES　　
# (3) Delete cookie s
response.delete_cookie("cookie_key",path="/",domain=name)

 

8.2.2,session

Django provides full support for anonymous sessions. This session framework allows you to store and retrieve any data of visitors to each site. It stores data on the server side, and sends and receives data in the form of cookies.

(1) session flow chart

(2) session grammar and cases

# 1. Set Sessions value
          request.session['session_name'] ="admin"
# 2. Get Sessions value
          session_name = request.session["session_name"]
# 3. Delete Sessions value
          del request.session["session_name"]
# 4,flush()
     # Delete the current session data and delete the session cookies. This is used to ensure that the previous session data cannot be accessed by the user's browser again

 

 

def s_login(request):
    if request.method == "GET":
        return render(request, "login.html")
    else:
        user = request.POST.get("user")
        pwd = request.POST.get("pwd")
        try:
            # user_obj = User.objects.get(user=user,pwd=pwd)
            # Write session
            # request.session["is_login"] = True
            # request.session["username"] = user_obj.user
            return redirect("/s_index/")
        except:
            return redirect("/s_login/")

          
def s_index(request):
    # Read session
    is_login = request.session.get("is_login")
    if is_login:
        username = request.session.get("username")
        return render(request, "index.html", {"user": username})
    else:
        return redirect("/s_login/")
'''
shop.html:
<p>
Last access time of client:{{ last_time|default:"First visit" }}
</p>
<h3>Product page</h3>
'''

def shop(request):
    last_time = request.session.get("last_time")
    now = datetime.datetime.now().strftime("%Y-%m-%d %X")
    request.session["last_time"] = now

    return render(request, "shop.html", {"last_time": last_time})


def s_logout(request):
    # request.session.flush()
    del request.session["username"]
    del request.session["is_login"]
    return redirect("/s_login/")

1. The session is on the server side and the cookie is on the client (browser)
2. The session is stored in a file (not memory) on the server by default
3. The operation of a session depends on the session id, which is stored in a cookie
4. Sessions can be placed in files, databases, or memory.
5. session is usually used for user authentication

(3) session configuration

# Django supports Session by default, and stores Session data in the database by default, that is, Django_ In the Session table.
   
# Configure settings py
   
    SESSION_ENGINE = 'django.contrib.sessions.backends.db'   # Engine (default)
       
    SESSION_COOKIE_NAME ＝ "sessionid"               # The key when the Session cookie is saved on the browser, that is, sessionid = random string (default)
    SESSION_COOKIE_PATH ＝ "/"                               # The path where Session cookie s are saved (default)
    SESSION_COOKIE_DOMAIN = None                             # Session cookie saved domain name (default)
    SESSION_COOKIE_SECURE = False                            # Whether Https transfer cookie s (default)
    SESSION_COOKIE_HTTPONLY = True                           # Whether Session cookie s only support http transmission (default)
    SESSION_COOKIE_AGE = 1209600                             # Session's cookie expiration date (2 weeks) (default)
    SESSION_EXPIRE_AT_BROWSER_CLOSE = False                  # Whether to close the browser to expire the Session (default)
    SESSION_SAVE_EVERY_REQUEST = False                       # Whether to save the Session for each request. By default, it can only be saved after modification (default)

8.2.3 user authentication component

Django has provided the auth module of the authentication system by default. When we authenticate, we will use the table provided to us in the auth module. The certification system includes:

• user management
• jurisdiction
• User group
• Password hashing system
• Forms and views for user login or content display
• A pluggable background system admin

(1) Django user model class

Django authentication system provides a User model class User to save User data. The default User contains the following common basic fields:

Some fields are missing from the above, so we will modify the current built-in user model later. For example, there is no mobile number field in it, and we need to add it later.

(2) Important methods

Django user authentication (auth) component needs to import auth module

# Authentication module
from django.contrib import auth
# The corresponding database user table can inherit and expand
from django.contrib.auth.models import User

(1) User object

create() # To create an ordinary user, the password is clear text.
create_user() # Create an ordinary user whose password is encrypted.
create_superuser() # And create_user() is the same, but is is set_ Staff and is_superuser is True.

set_password(*raw_password*)
# Set the User's password to the given original string and be responsible for the password. The User object is not saved. When None is raw_password, the password will be set to an unavailable password.
check_password(*raw_password*)
# If given raw_ If password is the user's real password, it returns True, which can be used when verifying the user's password.

(2) Certification method

auth.authenticate(username,password) 
# Turn the entered password into ciphertext to authenticate. If the authentication succeeds, the user object will be returned. If it fails, None will be returned

(3) Login and logout methods

from django.contrib import auth
# This function accepts an HttpRequest object and an authenticated User object. This function uses django's session framework to attach session id and other information to an authenticated User.
auth.login() 
# This function accepts an HttpRequest object with no return value. When this function is called, all the currently requested session information will be cleared. Even if the user is not logged in, there will be no error when using this function.
auth.logout() 

(4)request.user

(5) Customize user table

from django.contrib.auth.models import AbstractUser

Set the user model used by Auth authentication module to our own user model

Format: "sub application directory name. Model class name"

AUTH_USER_MODEL = 'users.User'

8.3 pager of Django

(1) index view

def index(request):
    '''
    Batch import data:

    Booklist=[]
    for i in range(100):
        Booklist.append(Book(title="book"+str(i),price=30+i*i))
    Book.objects.bulk_create(Booklist)



    Use of pager:

    book_list=Book.objects.all()

    paginator = Paginator(book_list, 10)

    print("count:",paginator.count)           #Total data
    print("num_pages",paginator.num_pages)    #PageCount 
    print("page_range",paginator.page_range)  #List of page numbers



    page1=paginator.page(1) # Page object of page 1
    for i in page1:         # Traverse all data objects on page 1
        print(i)

    print(page1.object_list) #All data on page 1


    page2=paginator.page(2)

    print(page2.has_next())            #Is there a next page
    print(page2.next_page_number())    #Page number of next page
    print(page2.has_previous())        #Is there a previous page
    print(page2.previous_page_number()) #Page number of previous page



    # Throw wrong
    #page=paginator.page(12)   # error:EmptyPage

    #page=paginator.page("z")   # error:PageNotAnInteger

    '''

    book_list = Book.objects.all()

    paginator = Paginator(book_list, 10)
    page = request.GET.get('page', 1)
    current_page = int(page)

    try:
        print(page)
        book_list = paginator.page(page)
    except PageNotAnInteger:
        book_list = paginator.page(1)
    except EmptyPage:
        book_list = paginator.page(paginator.num_pages)

    return render(request, "index.html", {"book_list": book_list, "paginator": paginator, "currentPage": current_page})

(2) index.html

<!DOCTYPE html>
<html lang="en">
<head>
    <meta charset="UTF-8">
    <title>Title</title>
    <link rel="stylesheet" href="https://cdn.bootcss.com/bootstrap/3.3.7/css/bootstrap.min.css" 
    integrity="sha384-BVYiiSIFeK1dGmJRAkycuHAHRg32OmUcww7on3RYdg4Va+PmSTsz/K68vbdEjh4u" crossorigin="anonymous">
</head>
<body>

<div class="container">

    <h4>Pager</h4>
    <ul>

        {% for book in book_list %}
             <li>{{ book.title }} -----{{ book.price }}</li>
        {% endfor %}

     </ul>


    <ul class="pagination" id="pager">

                 {% if book_list.has_previous %}
                    <li class="previous"><a href="/index/?page={{ book_list.previous_page_number }}">previous page</a></li>
                 {% else %}
                    <li class="previous disabled"><a href="#"> previous page</a></li>
                 {% endif %}


                 {% for num in paginator.page_range %}

                     {% if num == currentPage %}
                       <li class="item active"><a href="/index/?page={{ num }}">{{ num }}</a></li>
                     {% else %}
                       <li class="item"><a href="/index/?page={{ num }}">{{ num }}</a></li>

                     {% endif %}
                 {% endfor %}



                 {% if book_list.has_next %}
                    <li class="next"><a href="/index/?page={{ book_list.next_page_number }}">next page</a></li>
                 {% else %}
                    <li class="next disabled"><a href="#"> next page</a></li>
                 {% endif %}

            </ul>
</div>



</body>
</html>

8.4 FBV and CBV

1 FBV : function based view

2 BCV: class based view

8.4.1. Front and rear end separation mode

In developing Web applications, there are two application modes:

1. The front and back ends are not separated [the content seen by the client and all interface effects are provided by the server.]

1. Separation of front end and back end [separate the interface effect of the front end (html, css, js to another server, python server only needs to return data)]

The front end forms an independent website, and the service end forms an independent website

8.4.2 api interface

Application Programming Interface (API interface) is an external access to operation data provided by an application program. This access can be a function or class method, a url address or a network address. When the client calls this entry, the application will execute the corresponding code operations to complete the corresponding functions for the client.

Of course, api interface is a common development content in work. Sometimes, we call api interfaces written by others, and sometimes, we also need to provide api interfaces for others to operate. This will bring a problem. The api interface is often a function, class method, or url or other network address. Which one is it? When writing the api interface, we should all consider how to write this interface? How to write the interface is easier to maintain and clear, which requires you to have a clear writing specification when calling or writing api interfaces!!!

In order to form a consensus within the team and prevent confusion caused by differences in personal habits, we all need to find a good interface implementation specification, which can make the purpose of the interface written by the back end clear at a glance and reduce the cooperation cost between the client and the server.

At present, the interface implementation specifications used by most company developers in the market mainly include: restful and RPC.

RPC (Remote Procedure Call): translate into Chinese: Remote Procedure Call [remote service call] Literally, it means to access / call the api interface provided by the remote server. This kind of interface is generally provided by service or procedural code.

• The server provides a unique access entry address: http://api.xxx.com/ Or http://www.xx.com/api Or address based on other agreements

• When the client requests the server, all operations are understood as actions. Generally, in web development, the corresponding is the post request of HTTP request

• Specify the name of the interface to be called and the parameters required by the interface through the request body parameters

  action=get_all_student&class=301&sex=1

  m=get_all_student&sex=1&age=22

  command=100&sex=1&age=22

With more rpc interfaces, there are more corresponding function names and parameters, and the front end is difficult to find when requesting api interfaces The old rpc server code is also prone to duplicate interfaces

restful: translate into Chinese: resource state transition (representational state transition)

• If all the data / files provided by the server are regarded as resources, the operation of requesting data through the api interface is essentially the operation of resources

  Therefore, restful requires us to write the name of the resource in the url address when we operate on the resource provided by the current interface.

• In web development, the most common and common way to operate resources is to add, delete, check and modify, so restful requires that you state what resources to operate in the address bar. Then the http request verb is used to explain what kind of operation is performed on the resource

  POST http://www.xxx.com/api/students/ Add student data

  GET http://www.xxx.com/api/students/ Get all students

  GET http://www.xxx.com/api/students/1/ Get students with id=pk

  DELETE http://www.xxx.com/api/students/1/ Delete a student with id=pk

  PUT http://www.xxx.com/api/students/1/ Modify all the information of a student [id,name,sex,age,]

  PATCH http://www.xxx.com/api/students/1/ Modify some information of a student [age]

In other words, we only need to combine the resource name on the url address with the HTTP request action to explain the function of the current api interface. restful is a resource based api interface specification, which is reflected in the address that resources are expressed in terms of nouns. rpc is an api interface specification based on action, which is reflected in the action with operation data on the interface name.

8.4.3 RESTful API specification

The full name of REST is representative state transfer, which means expressive state transfer in Chinese. It first appeared in Roy Fielding's doctoral thesis in 2000.

RESTful is a design style that defines API interfaces specifically for Web development, especially in the application mode of front-end and back-end separation.

The concept of this style believes that the back-end development task is to provide data, and the external provision is the access interface of data resources. Therefore, when defining the interface, the URL path accessed by the client represents this data resource to be operated.

For data resources, POST, DELETE, GET, UPDATE and other request actions are used to express the addition, deletion, query and modification of data.

Restful specification is a general specification that does not restrict the use of languages and development frameworks. In fact, we can use any language and any framework can implement API interfaces that conform to restful specifications.

Reference documents: http://www.runoob.com/w3cnote/restful-architecture.html

In the process of interface implementation, idempotency will exist. Idempotency refers to whether different results will be generated for the resources in the server when the client initiates the same request many times. If the result of the server is the same after multiple requests, it belongs to idempotent interface. If the result of the server is different after multiple requests, it belongs to non idempotent interface.

8.4.4 CBV use

The view function we used before is called FBV (that is, functional view function). Here we will try to write CBV (class view function). Class view functions can make the code look simpler and more convenient to use.

# FBV
# def index(request):
#     if request.method == "GET":
#
#         return HttpResponse("GET")
#     elif request.method == "POST":
#
#         return HttpResponse("POST")
#
#     elif request.method == "DELETE":
#         return HttpResponse("DELETE")

# CBV mode: restful based development

class IndexView(View):

    def get(self, request):
        return HttpResponse("CBV GET")

    def post(self, request):
        return HttpResponse("CBV POST")


class BookView(View):

    def get(self, request):
        # get data
        book_list = Book.objects.all()

        # Serialization: json
        data_json = serializers.serialize("json", book_list)

        return HttpResponse(data_json, content_type="json")

# FBV mode
# path('index/', views.index),
# CBV mode
path("index/",views.IndexView.as_view()),
path("books/",views.BookView.as_view())

 

8.5. csrftoken (Cross Site Request Forgery)

CSRF (Cross Site Request Forgery) is a network attack method. This attack can forge a request in the name of the victim and send it to the attacked site without the victim's knowledge, so as to perform operations under permission protection without authorization, which is very harmful. Specifically, CSRF attack can be understood as follows: the attacker embezzles your identity and sends a malicious request in your name. For the server, this request is completely legal, but it completes an operation expected by the attacker, such as sending email and messages in your name, stealing your account, adding system administrators, and even purchasing goods, virtual currency transfer, etc.

-- Corporate email
-- pornhub.com

Reference article

A token is actually a token used for user authentication. The birth of a token cannot be separated from CSRF. It is precisely because CSRF will appear in the above Cookie/Session state retention mode, so there is a token.

• Uncomment Middleware

 

 

• No CSRF_ post request of token data

8.5.1 basic use

1, form submission

Add {% csrf_token%} directly to the html page form

! [screenshot 2021-10-31 10.30.29 PM] (assets/ screenshot 2021-10-31 10.30.29 PM. PNG)

2, ajax submission

Method 1: put it in the request data.

$.ajax({
  url: '/csrf_test/',
  method: 'post',
  data: {'name': $('[name="name"]').val(),
         'password': $('[name="password"]').val(),
         'csrfmiddlewaretoken':$('[name="csrfmiddlewaretoken"]').val()           
        },
  success: function (data) {
    console.log('succeed')
    console.log(data)           },
})

Method 2: put it in the request header

$.ajax({
            url: '/csrf_test/',
            method: 'post',
            headers:{'X-CSRFToken':'token value'},  // Pay attention to put it in quotation marks
            data:{}
}

8.5.2. Global use, local prohibition of csrf

(1) Add decorator to view function

from django.views.decorators.csrf import csrf_exempt,csrf_protect
@csrf_exempt
def Function name(request):  # After adding the decorator, there is no csrf verification for this view function

(2) View class

from django.views.decorators.csrf import csrf_exempt,csrf_protect
from django.utils.decorators import method_decorator
@method_decorator(csrf_exempt,name='dispatch')
class index(View):
    def get(self,request):
        return HttpResponse("GET")
    def post(self,request):
        return HttpResponse("POST")

 

8.6 advanced ORM

8.6.1 queryset feature

(1) Slicable

Use Python's slicing syntax to LIMIT the number of query set records. It is equivalent to the LIMIT and OFFSET clauses of SQL.

>>> Article.objects.all()[:5]   # (LIMIT 5)
>>> Article.objects.all()[5:10]    # (OFFSET 5 LIMIT 5)

Negative indexes (such as Article.objects.all()[-1]) are not supported. Typically, the slice of a query set returns a new query set -- it does not execute the query.

(2) Iteratable

articleList=models.Article.objects.all()

for article in articleList:
    print(article.title)

3) Lazy query

Query sets are executed lazily -- creating query sets does not bring any database access. You can keep the filter all day long, and Django won't really run the query until the query set needs to be evaluated.

queryResult=models.Article.objects.all() # not hits database
print(queryResult) # hits database
for article in queryResult:
    print(article.title)    # hits database

 

Generally speaking, only when "requesting" the results of query sets will they be obtained in the database. When you really need results, query sets are evaluated by accessing the database. About the exact time the evaluation occurred.

(4) Caching mechanism

Each query set contains a cache to minimize access to the database. Understanding how it works will allow you to write the most efficient code.

In a newly created query set, the cache is empty. The query set is evaluated for the first time - database queries occur at the same time - Django will save the results of the query into the cache of the query set and return the explicitly requested results (for example, if the query set is being iterated, the next result will be returned). The next evaluation of the query set will reuse the cached results.

Please keep this caching behavior in mind, because if you use the query set improperly, it will fool you. For example, the following statement creates two query sets, evaluates them, and then throws them away:

Note: simply printing a query set does not fill the cache.

This means that the same database query will be executed twice, obviously doubling your database load. At the same time, it is also possible that the two result lists do not contain the same database records, because articles may be added or deleted during the two requests. To avoid this problem, just save the query set and reuse it:

queryset = Book.objects.all()
ret = [i for i in queryset] # hit database
print(queryset) # Use cache
print(queryset) # Use cache

 

When are query sets cached?

1. When traversing queryset
2. if statement (to avoid this, you can use the exists() method to check whether there is data)

Therefore, the index or slice of a separate queryset will not be cached.

queryset = Book.objects.all()
one = queryset[0] # hit database
two = queryset[1] # hit database
print(one)
print(two)

(5) exists() and iterator() methods

exists

Simply using the if statement to judge will also completely execute the entire queryset and put the data into the cache, although you don't need these data! To avoid this, you can use the exists() method to check whether there is data:

 if queryResult.exists():
    #SELECT (1) AS "a" FROM "blog_article" LIMIT 1; args=()
        print("exists...")

iterator

When queryset is very large, cache will become a problem.

When processing thousands of records, it is a waste to load them into memory at one time. To make matters worse, the huge queryset may lock the system process and make your program close to crashing. To avoid generating queryset cache while traversing data, you can use the iterator() method to obtain data and discard it after processing.

objs = Book.objects.all().iterator()
# iterator() can get only a small amount of data from the database at a time, which can save memory
for obj in objs:
    print(obj.title)
#BUT, traversal again did not print, because the iterator has been traversed from the last time (next) to the last time, so there is no need to traverse
for obj in objs:
    print(obj.title)

Of course, using the iterator() method to prevent cache generation means that queries will be executed repeatedly when traversing the same queryset. So be careful when # using iterator(), and make sure that your code does not repeatedly execute queries when operating a large queryset.

The cache of queryset is used to reduce the query of the program to the database. Under normal use, it will ensure that the database will be queried only when necessary. Using the exists() and iterator() methods can optimize the memory usage of the program. However, because they do not generate queryset cache, additional database queries may be caused. 　

Summary: in the choice of using cache mechanism or generator mechanism, if yes, generator is mainly used in case of large amount of data; Cache mechanism is used when data is less and used more times.

8.6.2 intermediary model

When dealing with simple many to many relationships such as matching pizza and topping, it is OK to use the standard ManyToManyField. However, sometimes you may need to correlate data to the relationship between the two models.

For example, there is an application that records the music group to which the musician belongs. We can use a ManyToManyField to represent the many to many relationship between groups and members. However, sometimes you may want to know more details about membership, such as when members joined the group.

For these cases, Django allows you to specify a mediation model to define many to many relationships. You can put other fields in the mediation model. The ManyToManyField field of the source model will use the through parameter to point to the mediation model. For the above example of music group, the code is as follows:

from django.db import models
 
class Person(models.Model):
    name = models.CharField(max_length=128)
 
    def __str__(self):              # __unicode__ on Python 2
        return self.name
 
class Group(models.Model):
    name = models.CharField(max_length=128)
    members = models.ManyToManyField(Person, through='Membership')
 
    def __str__(self):              # __unicode__ on Python 2
        return self.name
 
class Membership(models.Model):
    person = models.ForeignKey(Person)
    group = models.ForeignKey(Group)
    date_joined = models.DateField()
    invite_reason = models.CharField(max_length=64)

Now that you have set up ManyToManyField to use the mediation model (in this case, Membership), you need to start creating many to many relationships. What you need to do is to create an instance of the mediation model:

>>> ringo = Person.objects.create(name="Ringo Starr")
>>> paul = Person.objects.create(name="Paul McCartney")
>>> beatles = Group.objects.create(name="The Beatles")
>>> m1 = Membership(person=ringo, group=beatles,
...     date_joined=date(1962, 8, 16),
...     invite_reason="Needed a new drummer.")
>>> m1.save()
>>> beatles.members.all()
[<Person: Ringo Starr>]
>>> ringo.group_set.all()
[<Group: The Beatles>]
>>> m2 = Membership.objects.create(person=paul, group=beatles,
...     date_joined=date(1960, 8, 1),
...     invite_reason="Wanted to form a band.")
>>> beatles.members.all()
[<Person: Ringo Starr>, <Person: Paul McCartney>]

Unlike ordinary many to many fields, you cannot use add, create, and assignment statements (for example, beats.members = [...]) To create a relationship:

# THIS WILL NOT WORK
>>> beatles.members.add(john)
# NEITHER WILL THIS
>>> beatles.members.create(name="George Harrison")
# AND NEITHER WILL THIS
>>> beatles.members = [john, paul, ringo, george]

Why not? This is because you cannot only create the association relationship between Person and Group, but also specify all the information required in the Membership model; Simple add, create and assignment statements cannot do this. Therefore, they cannot be used in many to many relationships using the mediation model. At this point, the only way is to create an instance of the mediation model.

The remove() method is disabled for the same reason. But the clear() method is available. It can clear all the many to many relationships of an instance:

>>> # Beatles have broken up
>>> beatles.members.clear()
>>> # Note that this deletes the intermediate model instances
>>> Membership.objects.all()
[]

8.6.3. Database table reverse generation model class

As we all know, Django is more suitable for native development, that is, to build a new project through the framework, by modifying models Py to create a new database table. But sometimes, we need to use the previously designed database, which provides a variety of designed forms. At this time, if we pass models Py will waste a lot of time designing again. Fortunately, Django provided us with the method of inspectdb. Its function is to reverse map the structure to models according to the existing mysql database tables Py

Before we show the reverse generation of django ORM, let's talk about how to generate code in the forward direction.

Forward generation refers to creating a model first Py file, and then through django's built-in compiler, create a model in a database such as mysql Py.

Reverse generation refers to creating a table in the database first, and then generating model code through django's built-in compiler.

 

python manage.py inspectdb > models file name

8.6.4 query optimization

(1)exists()

Judge whether there is data in the query set. If there is, return True; if not, return False

(2)select_related()

For OneToOneField and ForeignKey fields, select can be used_ Related to optimize QuerySet.

select_related returns a QuerySet. When executing its query, it queries the data of the associated object along the foreign key relationship. It will generate a complex query and cause performance loss, but database queries will not be required when using foreign key relationships in the future.

To put it simply, use select on QuerySet_ After the related () function, Django will get the object corresponding to the corresponding foreign key, so it is unnecessary to query the database later when it is needed.

The following example explains the common query and select_ The difference between related () queries.

Query the publisher name of the book with id=2. Here is a standard query:

# Hits the database.
book= models.Book.objects.get(nid=2)
# Hits the database again to get the related Blog object.
print(book.publish.name)

If we use select_related() function:

Multi foreign key query

This is a foreign key query for publish. What about another foreign key? Let's have a look:

book=models.Book.objects.select_related("publish").get(nid=1)
print(book.authors.all())

After observing the logging results, it is found that it still needs to be queried twice, so it needs to be changed to:

book=models.Book.objects.select_related("publish","").get(nid=1)
print(book.publish)

Or:

book=models.Article.objects
　　　　　　　　　　　　　.select_related("publish")
　　　　　　　　　　　　　.select_related("")
　　　　　　　　　　　　　.get(nid=1)  # django 1.7 supports chain operation
print(book.publish)

(3)prefetch_related()

For ManyToManyField and one to many fields, prefetch can be used_ Related() to optimize.

prefetch_related() and select_related() is designed to reduce the number of SQL queries, but it is implemented in different ways. The latter solves problems in SQL queries through JOIN statements. But for many to many relationships, using SQL statements to solve them seems a little unwise, because the tables obtained by joining will be very long, which will lead to the increase of SQL statement running time and memory occupation. If there are n objects, and the many to many fields of each object correspond to Mi bars, it will be generated Σ (n) Result table of row Mi.

prefetch_ The solution of related () is to query each table separately, and then use Python to deal with the relationship between them.

# Query all tags associated with all articles
books=models.Book.objects.all()
for book in books:
    print(book.authors.all())  #4 articles: hits database 5

Change to prefetch_related:

# Query all tags associated with all articles
books=models.Book.objects.prefetch_related("authors").all()
for book in books:
    print(book.authors.all())  #4 articles: hits database 2

(4)extra

extra(select=None, where=None, params=None, 
      tables=None, order_by=None, select_params=None)

In some cases, Django's query syntax is difficult to simply express complex WHERE clauses. In this case, Django provides an extra() QuerySet modification mechanism - it can inject new clauses into SQL clauses generated by QuerySet

extra can specify one or more parameters, such as select, where or tables These parameters are not necessary, but you must use at least one! Note that these additional methods may have portability problems for different database engines (because you are explicitly writing SQL statements), unless you have to, try to avoid this.

select of parameters

The select parameter allows you to add other field information to the select clause. It should be a dictionary that stores the mapping from attribute names to SQL clauses.

queryResult=models.Article
　　　　　　　　　　　.objects.extra(select={'is_recent': "create_time > '2017-09-05'"})

Each Entry object in the result set has an additional attribute is_recent, which is a Boolean value indicating the create of the Article object_ Whether the time is later than 2017-09-05

where / tables of parameters

You can use where to define explicit SQL where clauses - perhaps to perform non explicit joins. You can manually add tables to the SQL FROM clause using tables.

Both where and tables accept string lists. All the where parameters are "and" any other search criteria.

For example:

queryResult=models.Article
　　　　　　　　　　　.objects.extra(where=['nid in (3，4) OR title like "py%" ','nid>2'])

8.7 uploading documents

8.7.1 form form upload file

def index(request):
    return render(request,"index.html")

def upload_file(request):
    print("FILES:",request.FILES)
    print("POST:",request.POST)
    return HttpResponse("Upload successful!")

8.7.2 Ajax (based on FormData)

What is FormData?

XMLHttpRequest Level 2 adds a new interface FormData Using the FormData object, we can simulate a series of form controls with some key value pairs through JavaScript, and we can also use the send() method of XMLHttpRequest to submit this "form" asynchronously Compared with ordinary ajax, the biggest advantage of using FormData is that we can upload a binary file asynchronously

Newer versions of all major browsers already support this object, such as Chrome 7+, Firefox 4+, IE 10+, Opera 12+, and Safari 5+.

<h3>Ajax Upload file</h3>

<p><input type="text" name="username" id="username" placeholder="username"></p>
<p><input type="file" name="upload_file_ajax" id="upload_file_ajax"></p>

<button id="upload_button">Submit</button>
{#be careful button Do not use labels on form Use in form#}

<script>
    $("#upload_button").click(function(){

        var username=$("#username").val();
        var upload_file=$("#upload_file_ajax")[0].files[0];

        var formData=new FormData();
        formData.append("username",username);
        formData.append("upload_file_ajax",upload_file);


        $.ajax({
            url:"/upload_file/",
            type:"POST",
            data:formData,
            contentType:false,
            processData:false,

            success:function(){
                alert("Upload successful!")
            }
        });


    })
</script>

def index(request):
  
    return render(request,"index.html")
  
  
def upload_file(request):
    print("FILES:",request.FILES)
    print("POST:",request.POST)
    return HttpResponse("Upload successful!")

8.7.3 ImageField and FileField

ImageField and FileField can upload pictures and files to specified folders respectively.

1. In the following models Py:

picture = models.ImageField(upload_to= 'avatars/', default= "avatars/default.png", blank=True, null=True) Note: the parameter upload must be formulated when defining the ImageField field_ The relative path should be written in the to field,

This parameter will be added to settings Media in PY_ After the root, a path is formed, which is the storage location of the uploaded pictures. By default, it is under the Django project root path, that is, MEDIA_ROOT defaults to the Django root directory

So set mysite/settings first Py Media in PY_ ROOT

class Userinfo(models.Model):
    name = models.CharField(max_length=32)
    avatar_img = models.FileField("avatars/") 

username = request.POST.get("username")
#Get file object
file = request.FILES.get("file")   
#Insert data and assign the picture object directly to the field
user = Userinfo.objects.create(name=username,avatar_img=file)

Django will create an avatars folder in the root directory of the project, and download the uploaded file to this folder. The avatar field saves the relative path of the file.

1. On mysite/settings Py:

MEDIA_ROOT = os.path.join(BASE_DIR,"media")
MEDIA_URL='/media/'

MEDIA_ROOT: the path where media is stored. Add upload to this value_ The value of to is the location where the uploaded image file is actually stored

MEDIA_URL: after setting the value of this attribute, this value will be added in front of the link of the static file. If this value is set, userinfo avatar. The URL is automatically replaced with: /media/avatars/default Png, which can be called directly in the template:.

3.url.py:

from django.views.static import serve
# Add media configuration
re_path(r'^media/(?P<path>.*)$', serve, {'document_root': settings.MEDIA_ROOT}),

Browser can directly access http://127.0.0.1:8000/media/yuan/avatars/%E7%86%8A%E7%8C%AB.webp , that is, our users upload files.

Finally, I will add a user folder path to you

def user_directory_path(instance, filename):
    return os.path.join(instance.name,"avatars", filename)

class Userinfo(models.Model):
    name = models.CharField(max_length=32)
    avatar_img = models.FileField(upload_to=user_directory_path)  

4.FileField and imagefile are the same.

8.7.4 importing table batch creation data

# Create your tests here.
def multi_create(request):
    # Transfer the received excel file to mysql
    # (1) Download files to the server

    emp_excel = request.FILES.get("emp_excel")
    print(emp_excel) # Final topic (Shaanxi Unicom) xlsx
    print(emp_excel.name) # "Final topic (Shaanxi Unicom).xlsx"
    # Download files
    with open("files/"+emp_excel.name,"wb") as f:

        for line in emp_excel:
            f.write(line)


    # Read excel and import it into mysql in batch

    import os
    from openpyxl import load_workbook
    file_path = os.path.join("files",emp_excel.name)

    # Load an excel file
    wb = load_workbook(file_path)
    # Get sheet object
    print("wb.sheetnames",wb.sheetnames)
    worksheet = wb.worksheets[0]

    for row in worksheet.iter_rows(min_row=3):
        print(row)
        if row[0].value == None:
            break
        Employee.objects.create(name=row[0].value,
                                age=row[1].value,
                                ruzhi_date=row[2].value,
                                dep=row[3].value,
                                salary=row[4].value,
                                )



    return redirect("/index/")

8.8. Implementation of websocket

8.8.1 implementation of websocket

http request features:

Short link

Response based on request

TCP based

Stateless save

WebSocket is a protocol for full duplex communication over a single TCP connection. WebSocket allows the server to actively push data to the client. In WebSocket protocol, the client browser and the server only need to complete a handshake to create a persistent connection, and carry out two-way data transmission between the browser and the server.

In an HTTP access cycle, if you want to perform a long-time task, in order to avoid the browser waiting, the background must use asynchronous actions. At the same time, it also needs to meet the real-time needs. Users can visit the task details page at any time after submitting the task, where users can see the implementation progress of the task in real time.

For asynchronous task processing, we use celery to put tasks into the background for execution. Celery is a distributed asynchronous message task queue developed based on python, which can easily realize the asynchronous processing of tasks. Its use method is also mentioned in Netease Lede RDS design. When processing a task, celery will record the progress of the task in the database.

After realizing the background execution of the task, the next step is to solve the problem of updating the progress information to the web page in real time. From the previous step, we can know that the progress information of the task already exists in the database, and the web page can get the data by directly accessing the database. However, the update rate of progress information in the database is not fixed. If ajax polling with short interval is used to access the database, many useless requests will be generated, resulting in a waste of resources. Taking into consideration, we decided to use WebSocket to push task progress information. The website is built with Django, and the native MTV (model template view) design mode only supports Http requests. Fortunately, the Django development team has also seen the vigorous development of real-time network applications in the past few years, releasing Channels, which brings real-time capabilities to Django in the form of plug-ins. The following two figures show the native Django and the Django integrated with Channels.

Important fields in the request header of WebSocket:

Connection and Upgrade: indicates the WebSocket request initiated by the client

SEC WebSocket version: the WebSocket protocol version number used by the client. The server will confirm whether it supports this version number

SEC websocket key: a Base64 encoded value randomly generated by the browser and used to upgrade important fields in the response header of request WebSocket

Http/1.1 101 switching protocols: switching protocol. WebSocket protocol establishes TCP connection of transport layer through HTTP protocol

Connection and Upgrade: indicates the WebSocket response initiated by the server sec WebSocket accept: indicates that the server has accepted the request of the client, which is calculated by SEC WebSocket key

Advantages of WebSocket protocol:

It supports two-way communication, which is more real-time, light data format, low performance overhead, efficient communication, and supports expansion. Users can extend the protocol or implement customized sub protocols (such as supporting customized compression algorithms)

Disadvantages of WebSocket protocol:

A small number of browsers do not support it. There are differences in the degree and mode of browser support. Long connections require higher code stability for back-end processing services. The back-end push function is relatively complex. In the mature HTTP ecosystem, there are a large number of components that can be reused, and there are fewer WebSocket s

Application scenario of WebSocket:

Instant messaging, website message notification, online collaborative editing, such as Tencent document, multi player online games, video barrage, stock fund implementation quotation

8.8.2 Channels components

Channels has brought some new features to Django, the most obvious of which is the addition of WebSocket support. The most important part of channels can also be seen as a task queue. Messages are pushed to the channel by the producer and then delivered to one of the consumers listening to the channel. The main difference between it and the traditional task queue is that channels work through the network, so that producers and consumers can run transparently on multiple machines. This network layer is called channel layer. Channels recommends using redis as its channel layer backend, but you can also use other types of tools, such as rabbitmq, memory, or IPC. For some basic concepts of channels, it is recommended to read the official documents.

Inherit the connection of WebSocketConsumer.

AuthMiddlewareStack: used for WebSocket authentication, inheriting Cookie Middleware, SessionMiddleware, SessionMiddleware. django's channels encapsulate django's auth module. Using this configuration, we can obtain user information in the consumer through the following code

def connect(self):
    self.user = self.scope["user"]

self.scope is similar to the request in django. It contains the type, path, header, cookie, session, user and other useful information of the request

Configuration and use

channels==2.1.3

channels-redis==2.3.0.

The default port for starting Redis service is 6379. Django will use this port to connect to Redis service.

Update project configuration file settings Installed in PY_ Apps item

INSTALLED_APPS = [
    'django.contrib.admin',
    'django.contrib.auth',
    'django.contrib.contenttypes',
    'django.contrib.sessions',
    'django.contrib.messages',
    'django.contrib.staticfiles',
    'app01.apps.App01Config',
    'channels',
]

ASGI_APPLICATION = "chat.routing.application"

# WebSocket
CHANNEL_LAYERS = {
    'default': {
        'BACKEND': 'channels_redis.core.RedisChannelLayer',
        'CONFIG': {
            "hosts": [('127.0.0.1', 6379)],
        },
    },
}

from django.urls import path
from channels.auth import AuthMiddlewareStack
from channels.routing import ProtocolTypeRouter, URLRouter
from table.consumers import TableConsumer

application = ProtocolTypeRouter({
    # Empty for now (http->django views is added by default)
    'websocket': AuthMiddlewareStack(
        URLRouter([
            path('ws/table/<slug:table_id>/', TableConsumer),
        ])
    ),
})

routing.py route file and django url Py has similar functions and syntax, which means that the access to ws/table/ is handled by TableConsumer.

The new app is called table, and consumers are added under the table directory py:

from channels.generic.websocket import AsyncJsonWebsocketConsumer

class TableConsumer(AsyncJsonWebsocketConsumer):
    table = None

    async def connect(self):
        self.table = 'table_{}'.format(self.scope['url_route']['kwargs']['table_id'])
        # Join room group
        await self.channel_layer.group_add(self.table, self.channel_name)
        await self.accept()

    async def disconnect(self, close_code):
        # Leave room group
        await self.channel_layer.group_discard(self.table, self.channel_name)

    # Receive message from WebSocket
    async def receive_json(self, content, **kwargs):
        # Send message to room group
        self.from_client =  str(self.scope["client"])
        await self.channel_layer.group_send(self.table, {'type': 'message', 'message': content,"from_client":self.from_client})

    # Receive message from room group
    async def message(self, event):
        message = event['message']
        print("self.scope",self.scope)
        print("event",event)
        print("message",message)
        # Send message to WebSocket
        print(":::",event["from_client"]+">>>"+message)
        await self.send_json(event["from_client"]+">>>"+message)

The functions in the TableConsumer class are used to process the connection, disconnect, receive messages, and process the corresponding types of messages, where channel_ layer. group_ Send (self.table, {'type':'message','message': content}) method, self The table parameter is the group id of the current group. The {'type':'message','message': content} part is divided into two parts. The type is used to specify the type of the message. Different functions are called to process the message according to the message type, and the message body is inside the message.

Views in the table directory New functions in py:

from django.shortcuts import render

def table(request, table_id):
    return render(request, 'table.html', {
        'room_name_json': table_id
    })

from django.contrib import admin
from django.urls import path, re_path
from table.views import table

urlpatterns = [
    path('admin/', admin.site.urls),
    re_path('table/(\d+)', table),
]

Front end implementation WebSocket

WebSocket objects support four messages: onopen, onmessage, onclose, and oneror. Here we use two onmessage and onclose

Onopen: an onopen message will be triggered when the browser and websocket server are successfully connected

Oneror: if the connection fails, or the sending or receiving of data fails, or the data processing fails, the oneror message will be triggered

Onmessage: when the browser receives the data sent by the websocket server, it will trigger the onmessage message. The parameter e contains the data sent by the server

Onclose: when the browser receives the connection closing request sent by the websocket server, it will trigger the onclose message. Please indicate the source.

Add a new table in the templates\table directory of table html:

<!-- chat/templates/chat/room.html -->
<!DOCTYPE html>
<html>
<head>
    <meta charset="utf-8"/>
    <title>Chat Room</title>
</head>
<body>
<textarea id="chat-log" cols="100" rows="20"></textarea><br/>
<input id="chat-message-input" type="text" size="100"/><br/>
<input id="chat-message-submit" type="button" value="Send"/>
</body>
<script>
    var roomName = {{ room_name_json }};

    var chatSocket = new WebSocket('ws://' + window.location.host + '/ws/table/' + roomName + '/');

    chatSocket.onmessage = function (e) {
        var data = JSON.parse(e.data);
        document.querySelector('#chat-log').value += (JSON.stringify(data).slice(1,-1) + '\n');
    };

    chatSocket.onclose = function (e) {
        console.error('Chat socket closed unexpectedly');
    };

    document.querySelector('#chat-message-input').focus();
    document.querySelector('#chat-message-input').onkeyup = function (e) {
        if (e.keyCode === 13) {  // enter, return
            document.querySelector('#chat-message-submit').click();
        }
    };

    document.querySelector('#chat-message-submit').onclick = function (e) {
        var messageInputDom = document.querySelector('#chat-message-input');
        var message = messageInputDom.value;
        chatSocket.send(JSON.stringify(message));

        messageInputDom.value = '';
    };
</script>
</html>