Introduction and installation of Mongodb

learning target

1. Learn about the benefits of non-relational databases

2. Understand the installation of mongodb

1. Introduction to mongodb

1.1 What is mongodb

• mongodb is one of the most feature-rich NoSQL non-relational databases. Written in C++ language.

• mongodb itself provides S-side storage data, namely server; it also provides C-side operation processing (such as query, etc.) data, namely client.

1.2 The main difference between SQL and NoSQL

• Hierarchical relationship in SQL: Database > Table > Data

• In NoSQL, it is: Database > Collection > document

1.2.1 No correlation between data

• If you need to add external related data in SQL, the normalization method is to add a foreign key to the original table to associate the external data table.

• NoSQL can put external data directly into the original data set to improve query efficiency. The disadvantages are also more obvious, and it will be more troublesome to update the linked data.

• In SQL, the fields of each piece of data in a table are fixed. In NoSQL, the keys (fields) of each document (data) in a collection (table) can be different from each other.

1.2.2 Further reading

https://www.cnblogs.com/jeakeven/p/5402095.html

1.3 Advantages of mongodb as a non-relational database compared to relational databases

Easy to expand: There are many types of NoSQL databases, but a common feature is to remove the relational features of relational databases. There is no relationship between the data, so it is very easy to expand

Large data volume, high performance: NoSQL databases have very high read and write performance, especially in large data volumes. Thanks to its non-relational nature, the structure of the database is simple

Flexible data model: NoSQL does not need to establish fields for the data to be stored in advance, and can store customized data formats at any time. In relational databases, adding or deleting fields is a very troublesome thing. If it is a table with a very large amount of data, adding fields is a nightmare

2. mongodb installation

Take ubuntu18.04 as an example

mongodb has two installation methods: command installation or source installation

2.1 Command installation

Install using apt-get tool in ubuntu

sudo apt-get install -y mongodb-org

or refer to the official documentation https://docs.mongodb.com/manual/tutorial/install-mongodb-on-ubuntu/

2.2 Source installation

2.2.1 Select the corresponding version and operating system and download

https://www.mongodb.com/download-center/community?jmp=docs

2.2.2 Decompression

tar -zxvf mongodb-linux-x86_64-ubuntu1804-4.0.3.tgz

2.2.3 Move to /usr/local/ directory

sudo mv -r mongodb-linux-x86_64-ubuntu1804-4.0.3/ /usr/local/mongodb

2.2.4 Add the mongodb executable file to the environment variable PATH in the shell's initialization script .bashrc

a. Enter the .bashrc file

cd ~
sudo vi .bashrc

b. Add at the end of the .bashrc file:

export PATH=/usr/local/mongodb/bin:$PATH

3. Official documentation of mongodb

https://docs.mongodb.com/manual/introduction/

summary

1. Learn about the benefits of non-relational databases

   • Easy to expand

   • high performance

   • flexible data fields

2. Understand the installation of mongodb

   • sudo apt-get install -y mongodb-org

Simple use of mongodb

learning target

1. Master server startup

2. Master the use of the client

3. Master the database and collection commands of mongodb

4. Understanding the _id field in documents

1. Start the mongodb server

• Default port: 27017

• Location of the default configuration file: /etc/mongod.conf

• Default log location: /var/log/mongodb/mongod.log

There are two ways to start the mongodb server:

• Start of local test mode (only has the function of adding, deleting, modifying and checking local data)

• Production environment starts (with full full functionality)

1.1 Test mode start

• Start: sudo service mongod start (sudo service mongod start)

• Stop: sudo service mongod stop

• Restart: sudo service mongod restart

1.2 How to formally start the production environment

Start: sudo mongod --auth --dbpath=dbpath --logpath=logpath --append --fork

• When only starting with the sudo mongod command, the data is stored in the /data/db directory by default and needs to be created manually

• --dbpath: Specify the storage path of the database

• --logpath: specify the storage path of the log

• --append: or --logappend set the log writing form to append mode

• --fork: or -fork start a new process to run the mongodb service

• --f: or -f configuration file path (the above configuration information can be written into a file and then loaded and started through the parameters in the file)

• --auth: start with authorization authentication, we will learn this content in later courses

1.3 Check if the startup is successful

ps aux | grep mongod

2. Start the mongodb client: enter the mongo shell

• Start local client: mongo

• View help: mongo --help

• Exit: exit or ctrl+c

3. Simple use of mongodb

When the mongodb server is turned on, after entering the mongo shell, it can be used simply

3.1 Commands of mongodb database

• View the current database: db (the test database is used by default without switching databases)

• View all databases: show dbs /show databases

• Switch database: use db_name

  • db_name is the database name returned after show dbs

• Drop the current database: db.dropDatabase()

3.2 Commands of mongodb collection

• No need to manually create collections: collections are automatically created the first time you add data to a non-existing collection

• To create a collection manually:

  • db.createCollection(name,options)

  • db.createCollection("stu")

  • db.createCollection("sub", { capped : true, size : 10 } )

  • Parameter capped: The default value is false, which means no upper limit is set, and the value is true, which means setting the upper limit.

  • Parameters size: The number of bytes occupied by the collection. When the capped value is true, this parameter needs to be specified, indicating the upper limit size. When the document reaches the upper limit, the previous data will be overwritten, in bytes

• View collections: show collections

• Drop a collection: db.collectionname.drop()

• Check if the collection is capped: db.collectionname.isCapped()

3.3 Simple exercises

Enter the following command in the mongo shell to see the result

show dbs
use test
show collections
db
db.stu.insert({'name':'Guo Jing', 'age':22})
show dbs
show collections
db.stu.find()
db.stu.drop()
show collections
db.dropDatabase()
show dbs
exit

3.3 Common data types in mongodb (understand)

3.3.1 Common Types

• Object ID: document ID/data ID, data primary key

• String: String, most commonly used, must be valid UTF-8

• Boolean: stores a boolean value, true or false

• Integer: Integer can be 32-bit or 64-bit, depending on the server

• Double: floating point number

• Arrays: arrays/lists

• Object: A piece of data/document in mongodb, i.e. document nested document

• Null: store null value

• Timestamp: Timestamp, representing the total number of seconds since 1970-1-1

• Date: UNIX time format to store the current date or time

3.3.2 Notes

• Each document has an attribute, id, to ensure the uniqueness of each document, mongodb uses id as the primary key by default

  • The value of id can be set manually, if not provided, then MongoDB provides each document with a unique id of type objectID

• objectID is a 12-byte hexadecimal number, with two bits per byte, a total of 24-bit strings:

  • The first 4 bytes are the current timestamp

  • Next 3 bytes of machine ID

  • MongoDB's service process id in the next 2 bytes

  • The last 3 bytes are simple increment values

summary

1. server start

   • sudo mongod --dbpath=dbpath

2. Enter the mongo shell client

   • mongo

3. Database and collection commands for mongodb

   • show dbs

   • use db_name

   • show collections

   • db

   • db.collectionname.drop()

   • db.dropDatabase()

   • exit

4. Understanding the _id field in documents

Mongodb's CRUD

learning target

1. Master the method of inserting data in mongodb

2. Master the method of saving data in mongodb

3. Master the method of querying data in mongodb

4. Master the processing method of mongodb query results

5. Master the method of updating data in mongodb

6. Master the method of deleting data in mongodb

1. mongodb insert data

Command: db.collectionname.insert(document)

db.stu.insert({name:'gj', gender:1})
db.stu.insert({_id:"20170101", name:'gj', gender:1})

When inserting a document, if the _id parameter is not specified, MongoDB will automatically assign a unique ObjectId to the document

2. mongodb storage

Command: db.collectionname.save(document)

db.stu.save({_id:'20170101', name:'gj', gender:2})
db.stu.save({name:'gj', gender:2})
db.stu.find()

Modify if the id of the document already exists, add if the id does not exist

3 mongodb query

Command: db.collectionname.find()

The following data can be used to practice

db.stu.insert([{"name" : "Guo Jing", "hometown" : "Mongolia", "age" : 20, "gender" : true },
{"name" : "Huang Rong", "hometown" : "Peach Blossom Island", "age" : 18, "gender" : false },
{"name" : "Huazheng", "hometown" : "Mongolia", "age" : 18, "gender" : false },
{"name" : "Huang Yaoshi", "hometown" : "Peach Blossom Island", "age" : 40, "gender" : true },
{"name" : "Duan Yu", "hometown" : "Dali", "age" : 16, "gender" : true },
{"name" : "Duan Wangye", "hometown" : "Dali", "age" : 45, "gender" : true },
{"name" : "Hong Qigong", "hometown" : "Huazheng", "age" : 18, "gender" : true }])

3.1 Simple query

• Method find(): query

  db.collection name.find({condition document})

• Method findOne(): query, only return the first one

  db.collection name.findOne({condition document})

• Method pretty(): Formats the result; cannot be used with findOne()!

  db.collection name.find({conditional document}).pretty()

3.2 Comparison Operators

• Equal to: The default is equal to judgment, no operator

• Less than: $lt (less than)

• Less than or equal: $lte (less than equal)

• Greater than: $gt (greater than)

• Greater than or equal to: $gte

• Not equal to: $ne

Query all students older than 18
db.stu.find({age:{$gte:18}})

3.3 Logical Operators

Logical operators mainly refer to AND, OR logic

• and: write multiple conditions in json

Query age is greater than or equal to 18, and gender is true s student
db.stu.find({age:{$gte:18},gender:true})

or: use $or, the value is an array, and each element in the array is json

Query age is greater than 18, or gender is false s student
db.stu.find({$or:[{age:{$gt:18}},{gender:false}]})

The query age is greater than 18 or the gender is male, and the name is Guo Jing
db.stu.find({$or:[{age:{$gte:18}},{gender:true}],name:'gj'})

3.4 Range operators

Use $in, $nin to determine whether the data is in an array

Query students aged 18, 28
db.stu.find({age:{$in:[18,28,38]}})

3.5 Support for regular expressions

Writing regular expressions using $regex

Inquire name by'yellow'data at the beginning
db.stu.find({name:{$regex:'^yellow'}})

3.6 Custom Query

The mongo shell is a js execution environment that uses $where to write a function that returns data that meets the conditions

Query students older than 30
db.stu.find({
 $where:function() {
     return this.age>30;}
})

3.7 skip and limit

• Method limit(): used to read the specified number of documents

db.collection name.find().limit(NUMBER)
Query 2 pieces of student information
db.stu.find().limit(2)

Method skip(): used to skip a specified number of documents

db.collection name.find().skip(NUMBER)
db.stu.find().skip(2)

use simultaneously

db.stu.find().limit(4).skip(5)
db.stu.find().skip(5).limit(4)

Note: Using skip first is more efficient than using limit

3.8 Projection

In the returned results of the query, only the necessary fields are selected

Command: db.collection name.find({},{field name:1,...})

The parameters are fields and values, a value of 1 means display, and a value of 0 does not show special attention:

• For the _id column, it is displayed by default. If it is not displayed, it needs to be explicitly set to 0

• Cannot be set to 0 for other fields that are not displayed

db.stu.find({},{_id:0,name:1,gender:1})

3.9 Sorting

The method sort() is used to sort the query results according to the specified field

Command: db.setname.find().sort({field:1,...})

Parameter 1 is for ascending order parameter -1 is for descending order

Descending by gender, then ascending by age
db.stu.find().sort({gender:-1,age:1})

3.10 Statistics

The method count() is used to count the number of documents in the result set

Command: db.collection name.find({condition}).count() Command: db.collection name.count({condition})

db.stu.find({gender:true}).count()
db.stu.count({age:{$gt:20},gender:true})

4 mongodb update

db.collection name.update({query}, {update}, {multi: boolean})

- parameter query:Query conditions
- parameter update:update operator
- parameter multi:optional, default is false，Indicates that only the first data found is updated, and the value is true Indicates that all data that meets the conditions are updated

db.stu.update({name:'hr'},{name:'mnc'})           # Full document coverage update
db.stu.update({name:'hr'},{$set:{name:'hys'}})    # Specify the key-value update operation
db.stu.update({},{$set:{gender:0}},{multi:true})  # update all

Note: "multi update only works with $operators"

• The multi parameter must be used with $set!

5 mongodb delete

db.collection name.remove({query}, {justOne: boolean})

- parameter query:optional, conditions for deleted documents
- parameter justOne:optional, if set to true or 1, only delete one, the default false，means delete all

summary

1. Add db.collection name.insert({data})db.collection name.save({complete data including id}) in mongo shell # Save according to the specified id, update if it exists, insert if it does not exist

2. Delete db.collection name.remove({condition}, {justOne: true/false}) in mongo shell

3. Change db.set name in mongo shell.update({condition}, {$set:{complete data/partial fields}}, {multi: true/false})

4. Lookup db.collection name.find({condition}, {field projection}) in mongo shell

Aggregation operation of mongodb

learning target

1. Understand the aggregation principle of mongodb

2. Master the pipe command of mongdb

3. Master the expressions of mongdb

1 What is the aggregation of mongodb

Aggregate is an aggregation pipeline based on data processing. Each document passes through a pipeline consisting of multiple stages. The pipeline of each stage can be grouped, filtered and other functions, and then processed through a series of output. corresponding results.

Syntax: db.collection name.aggregate({pipeline:{expression}})

<img src= "./images/mongodb aggregation.Png" width = "100%" / >

2 Common pipes and expressions of mongodb

Knowledge points:

• Master the syntax of pipes in mongodb

• Master the pipeline command in mongodb

2.1 Common Pipeline Commands

In mongodb, after the document is processed, the next processing is performed through the pipeline. Common pipeline commands are as follows:

• $group: Group the documents in the collection, which can be used for statistical results

• $match: filter data, only output documents that meet the conditions

• $project: Modify the structure of the input document, such as renaming, adding, deleting fields, creating calculation results

• $sort: sort the input documents and output them

• $limit: Limit the number of documents returned by the aggregation pipeline

• $skip: Skip the specified number of documents and return the rest

2.2 Common expressions

Expression: Process input documents and output Syntax: Expression: '$column name' Common expressions:

• $sum: Calculate the sum, $sum:1 means double count

• $avg: Calculate the average

• $min: get the minimum value

• $max: get the maximum value

• $push: Insert values ​​into an array in the resulting document

3 $group of pipeline command

3.1 Group by a field

$group is the most used command among all aggregation commands. It is used to group documents in a collection and can be used for statistical results

The usage example is as follows

db.stu.aggregate(
    {$group:
        {
            _id:"$gender",
            counter:{$sum:1}
        }
    }
)

Among the points to note:

• db.db_name.aggregate is the syntax, all pipeline commands need to be written in it

• _id indicates the basis of grouping, according to which field to group, you need to use $gender to indicate that this field is selected for grouping

• $sum:1 means that each piece of data is counted as 1, and the count is the number of pieces of data under the group

3.2 group by null

When we need to count the entire document, another use of $group is to divide the entire document into a group for statistics

Examples of use are as follows:

db.stu.aggregate(
    {$group:
        {
            _id:null,
            counter:{$sum:1}
        }
    }
)

Among the points to note:

• _id:null means that no grouping fields are specified, that is, the entire document is counted, and the counter obtained at this time indicates the number of the entire document

3.3 Pivot

Under normal circumstances, when collecting data of different genders, you need to know all the names, and you need to observe them one by one. If you put all the names together in some way, then it can be understood as a data perspective at this time.

The usage example is as follows:

1. Statistics for students of different genders

   db.stu.aggregate(
       {$group:
           {
               _id:null,
               name:{$push:"$name"}
           }
       }
   )

Use $$ROOT to put the entire document into an array

 db.stu.aggregate(
     {$group:
         {
             _id:null,
             name:{$push:"$$ROOT"}
         }
     }
 )

3.4 Hands-on

For the following data, you need to count the number of userids in each country/province (the same userid is only counted once)

{ "country" : "china", "province" : "sh", "userid" : "a" }  
{  "country" : "china", "province" : "sh", "userid" : "b" }  
{  "country" : "china", "province" : "sh", "userid" : "a" }  
{  "country" : "china", "province" : "sh", "userid" : "c" }  
{  "country" : "china", "province" : "bj", "userid" : "da" }  
{  "country" : "china", "province" : "bj", "userid" : "fa" }

reference answer

db.tv3.aggregate(
  {$group:{_id:{country:'$country',province:'$province',userid:'$userid'}}},
  {$group:{_id:{country:'$_id.country',province:'$_id.province'},count:{$sum:1}}}

4 $match for pipeline commands

$match is used to filter data. It is a command that can be used in aggregation operations. The difference from find is that the $match operation can hand over the result to the next pipeline for processing, while find cannot.

The usage example is as follows:

1. Query students older than 20

   db.stu.aggregate(
       {$match:{age:{$gt:20}}
       )

Query the number of male and female students older than 20

 db.stu.aggregate(
     {$match:{age:{$gt:20}}
     {$group:{_id:"$gender",counter:{$sum:1}}}
     )

5 $project of pipeline command

$project is used to modify the input and output structure of the document, such as renaming, adding, deleting fields

The usage example is as follows:

1. Query the age and name of the student, output only the age and name

   db.stu.aggregate(
       {$project:{_id:0,name:1,age:1}}
       )

Query the life of boys and girls, and output the number of people

 db.stu.aggregate(
     {$group:{_id:"$gender",counter:{$sum:1}}}
     {$project:{_id:0,counter:1}}
     )

5.1 Hands-on exercises

For the following data: count the number of userids under each country/province (the same userid is counted only once), the fields in the result are {country:"",province:"",counter:"*"}

{ "country" : "china", "province" : "sh", "userid" : "a" }  
{  "country" : "china", "province" : "sh", "userid" : "b" }  
{  "country" : "china", "province" : "sh", "userid" : "a" }  
{  "country" : "china", "province" : "sh", "userid" : "c" }  
{  "country" : "china", "province" : "bj", "userid" : "da" }  
{  "country" : "china", "province" : "bj", "userid" : "fa" }  

reference answer

db.tv3.aggregate(
  {$group:{_id:{country:'$country',province:'$province',userid:'$userid'}}},
  {$group:{_id:{country:'$_id.country',province:'$_id.province'},count:{$sum:1}}},
  {$project:{_id:0,country:'$_id.country',province:'$_id.province',counter:'$count'}}
  )

6 $sort of pipeline command

$sort is used to sort the input documents and output them

The usage example is as follows:

1. Query student information in ascending order by age

   db.stu.aggregate({$sort:{age:1}})

Query the number of men and women, in descending order of number

 db.stu.aggregate(
     {$group:{_id:"$gender",counter:{$sum:1}}},
     {$sort:{counter:-1}}
 )

7 $skip and $limit of pipeline commands

• $limit limit the number of returned data

• $skip skips the specified number of documents and returns the number of documents left

• When using at the same time, use skip first and then use limit

The usage example is as follows:

1. Query 2 pieces of student information

   db.stu.aggregate(
       {$limit:2}
   )

Inquire about student information starting from Article 3

 db.stu.aggregate(
     {$skip:3}
 )

Count the number of boys and girls, in ascending order of the number, return the second data

 db.stu.aggregate(
     {$group:{_id:"$gender",counter:{$sum:1}}},
     {$sort:{counter:-1}},
     {$skip:1},
     {$limit:1}
 )

8 Summary

1. Understand what the aggregate operation is doing

2. Master the use of $group,$match,$project

3. Familiar with the use of $sort,$limit,$skip

4. Implement commonly used expressions

Mongodb index operation

learning target

1. Master the creation and deletion of mongodb indexes

2. Master the method of viewing indexes in mongodb

3. Master the method of creating a unique index in mongodb

1. Why does mongdb need to create an index

• Speed ​​up queries

• data deduplication

2. mongodb create a simple index method

• Syntax: db.collection name.ensureIndex({property:1}), 1 means ascending order, -1 means descending order

3. Comparison of query speed before and after index creation

Test: Insert 100,000 pieces of data into the database

Insert data:

for(i=0;i<100000;i++){db.t1.insert({name:'test'+i,age:i})}

Before creating the index:

db.t1.find({name:'test10000'})
db.t1.find({name:'test10000'}).explain('executionStats') # Displays details of query operations

Create an index:

db.t1.ensureIndex({name:1})

After creating the index:

db.t1.find({name:'test10000'}).explain('executionStats')

Speed ​​comparison before and after

<img src="./images/4.3.Create index speed comparison.png" width = "100%" />    

4. Viewing the index

By default _id is the index view method of the collection: db.collection name.getIndexes()

5. Delete the index

Syntax: db.collection name.dropIndex({'index name':1})

db.t1.dropIndex({name:1})
db.t1.getIndexes()

6. mongodb create unique index

By default, the value of the index field of mongdb can be the same. After creating a unique index, the database will check whether the value of the created index field exists when inserting data. If it exists, the data will not be inserted, but creating an index only It can improve the query speed and reduce the insertion speed of the database.

6.1 Syntax for adding a unique index:

db.collection name.ensureIndex({"field name":1}, {"unique":true})

6.2 Data deduplication using unique index

Based on the value of the field specified by the unique index, if the same, the data cannot be inserted

db.t1.ensureIndex({"name":1}, {"unique":true})
db.t1.insert({name: 'test10000'})

7. Create a composite index

When deduplicating data, a domain may be used to ensure the uniqueness of data. At this time, you can consider building a composite index to achieve this.

For example: grabbing the whole post bar information, it is not advisable to deduplicate the data by using the name of the post as a unique index, because there may be many posts with the same name

Syntax for building a composite index: db.collection_name.ensureIndex({field 1:1, field 2:1})

8. Points to note when building an index

• Choose whether you need to create a unique index according to your needs

• Whether the index fields are in ascending or descending order does not affect the query efficiency in the case of a single index, but it will affect the condition with a composite index

• The index needs to be created only when the amount of data is huge and the read operations of the database are very frequent. If the write operations are very frequent, creating an index will affect the writing speed.

  For example: when querying, if field 1 needs to be sorted in ascending order, and field 2 needs to be sorted in descending order, then the establishment of the composite index needs to set field 1 to 1 and field 2 to -1

After class thinking

Why should the database do read-write separation (the meaning of read-write separation)?

summary

1. Master the creation and deletion of mongodb indexes

   1. Master the method of viewing indexes in mongodb

      1. Master the method of creating a unique index in mongodb

Permission management in Mongodb

learning target

1. Understand the permission management of mongodb

1. Why do you need to set permissions management

The newly installed mongodb does not start with permission authentication by default. Unlike MySQL, mongodb does not have permissions set during installation. However, the public network operating system needs to set permissions to ensure data security, so we need to learn mongodb permission management

2. Permission management scheme of mongodb

• MongoDB does not have a default administrator account, so you need to add an administrator account first, and the mongodb server needs to open the authentication mode when running

  • The user can only log in to the database where the user is located (create the user's database), including the administrator account.

  • The administrator can manage all databases, but cannot directly manage other databases, only after authentication.

3. Creation of mongodb super administrator account

3.1 Create super user

enter the mongo shell

sudo mongod

Use the admin database (the super administrator account must be created on this database)

use admin

Create superuser

db.createUser({"user":"python","pwd":"python","roles":["root"]})

Successful creation will display the following information

Successfully added user: { "user" : "python", "roles" : [ "root" ] }

exit the mongo shell

exit

3.2 Start the mongodb database with permission authentication

sudo mongod --auth

After startup, there will be the following information in the startup information, indicating that mongodb is successfully started in the way of authorization authentication

[initandlisten] options: { security: { authorization: "enabled" } }

3.3 Login verification

At this point, when using the database commands again, a permission error will be reported, and authentication is required to perform the corresponding operation.

use admin
db.auth('python','python')

• The python user is created on the admin database, so you must come to the admin database for authentication

• It will return 1 if the authentication is successful, and 0 if it fails.

4. Create a common user

4.1 Create a normal user on the database used

1. Select the database where the user needs to be created

use test1

create user

db.createUser("user":"user1", "pwd":"pwd1", roles:["read"])
Create a normal user user1,This user is in test1 Permissions on are read only
db.createUser("user":"user1", "pwd":"pwd1", roles:["readWrite"])
Create a normal user user1,This user is in test1 Permissions on are read and write

4.2 Create a common user on the admin user database

use admin
db.createUser({"user":"python1", "pwd":"python1", roles:[{"role":"read","db":"dbname1"},{"role":"readWrite","db":"dbname2"}
]})

Create a python1 user on admin. The python1 user has two permissions, one is read-only on dbname1, and the other is read-write on dbname2

5. View created users

show users
{
	"_id" : "admin.python",
	"user" : "python",
	"db" : "admin",
	"roles" : [
		{
			"role" : "root",
			"db" : "admin"
		}
	]
}

6. Delete User

6.1 Enter the database where the account data is located

use db_name

6.2 Delete User

db.dropUser('python')

summary

1. Understand the permission management of mongodb

2. Familiarize yourself with the appropriate process for creating users

mongodb and python interaction

learning target

1. Master the method of adding, deleting, modifying and checking the interaction between mongdb and python

2. Master the way of authorization authentication using the pymongo module

1. The module that mongodb interacts with python

pymongo provides all the methods of interaction between mongdb and python: pip install pymongo

2. Using pymongo

2.1 Import pymongo and select the collection to operate

Databases and collections can be created automatically

2.1.1 Create connection objects and set operation objects without authorization authentication

from pymongo import MongoClient

client = MongoClient(host,port) # If it is a local connection host, the port parameter can be omitted

collection = client[db name][collection name]
# collection = client.db name.collection name # Same usage as above

2.1.2 Create connection objects and set operation objects in a way that requires authorization authentication

from pymongo import MongoClient
from urllib.parse import quote_plus

user = 'python' # account
password = 'python' # password
host = '127.0.0.1' # host
port = 27017 # port
uri = "mongodb://%s:%s@%s" % (quote_plus(user),
                              quote_plus(password),
                              host)
# quote_plus function: encode the url
# uri = mongodb://python:python@127.0.0.1
client = MongoClient(uri, port=port)
collection = client.db name.collection name

2.2 insert() to add data

insert can insert a list of data in batches, or insert a piece of data

collection.insert({a piece of data})
collection.insert([{data one},{data two}])

2.2.1 Add a piece of data

Returns the _id of the inserted data

ret = collection.insert({"name":"test10010","age":33})
print(ret)

2.2.2 Add multiple pieces of data

Returns a list of ObjectId objects

item_list = [{"name":"test1000{}".format(i)} for i in range(10)]
rets = collection.insert(item_list)
print(rets)
for ret in rets:
    print(ret)

2.3 find_one() to find a piece of data

Receive a condition in the form of a dictionary, and return the entire data in the form of a dictionary. If the condition is empty, return the first item

ret = client.test.test.find_one({'name': 'test10001'})
print(ret) # A dictionary containing the ObjectId objects of mongodb
_ = ret.pop('_id') # Clear the k,v of the ObjectId object of mongodb
print(ret) 

2.4 find() to find all data

Returns all the results that meet the conditions. If the conditions are empty, all the results returned are a Cursor cursor object, which is an iterable object, which can be similar to the pointer to read the file, but can only be read once

rets = collection.find({"name":"test10005"})，
for ret in rets:
    print(ret)
for ret in rets: #There is no content in rets at this time
    print(ret)

2.5 update() update data (overwrite the whole document or specify the key value, update one or more items)

• Syntax: collection.update({condition}, {'$set':{specified kv or a complete piece of data}}, multi=False/True, upsert=False/True)

• multi parameter: the default is False, which means to update one; multi=True to update multiple; the multi parameter must be used together with $set

• upsert parameter: the default is False; upsert=True, first query whether it exists, then update if it exists; if it does not exist, insert

• $set indicates that the specified field is updated

2.5.1 Update a piece of data; full document coverage; update if it exists, insert if it does not exist

data = {'msg':'This is a complete data 1','name':'Ha ha'}
client.test.test.update({'haha': 'heihei'}, {'$set':data}, upsert=True)

2.5.2 Update multiple pieces of data; full document coverage; update if it exists, and insert if it does not exist

data = {'msg':'This is a complete data 2','name':'Ha ha'} # The complete data is obtained after querying
client.test.test.update({}, {'$set':data}, multi=True, upsert=True)

2.5.3 Update a piece of data; specify the key value; update if it exists, insert if it does not exist

data = {'msg':'Specifies to update only msg___1'}
client.test.test.update({}, {'$set':data}, upsert=True)

2.5.4 Update multiple pieces of data; specify the key value; update if it exists, or insert if it does not exist

data = {'msg':'Specifies to update only msg___2'}
client.test.test.update({}, {'$set':data}, multi=True, upsert=True)

2.6 delete_one() deletes a piece of data

collection.delete_one({"name":"test10010"})

2.7 delete_many() delete all data

collection.delete_many({"name":"test10010"})

3. Other APIs of the pymongo module

View pymongo official documentation or source code http://api.mongodb.com/python/current/

summary

1. Master the use of pymongo's addition, deletion, modification and query

2. Use the pymongo module to master the way of authorization authentication