Ten thousand words long text | get started with Go language - Basic language | notes of the third byte beating youth training camp

Day01 go language - Basic language - Notes of youth training camp

The reading effect on the PC is better. The source code has been uploaded here:

Source code: https://github.com/nateshao/gogogo/tree/master/day01-05-07

  • Day01 go language - Basic language - Notes of youth training camp
  • 1.1 what is Go language
  • 1.2 which companies use Go language
  • 1.3 why byte beating embraces GO language
  • 2.1 development environment - install Golang
  • 2.1 development environment - configure the integrated development environment
  • 2.2 basic grammar Hello World
  • 2.2 basic syntax - variables
  • 2.3 basic syntax -if else
  • 2.4 basic syntax - loop
  • 2.5 basic grammar switch
  • 2.6 basic syntax - array
  • 2.7 basic grammar - Slicing
  • 2.8 basic syntax map
  • 2.9 basic grammar range
  • 2.10 basic syntax - functions
  • 2.11 basic syntax - pointer
  • 2.12 basic grammar - structure
  • 2.13 basic grammar - structure method
  • 2.14 basic syntax - error handling
  • 2.15 basic syntax - string operation
  • 2.16 basic syntax - string formatting
  • 2.17 basic syntax JSON processing
  • 2.18 basic grammar - time processing
  • 2.19 basic grammar - number parsing
  • 2.20 basic syntax - process information
  • 3.1 introduction to guessing game
  • 3.1.1 guessing game - generate random numbers
  • 3.1.2 guessing game - generate random number effect
  • 3.1.2 guessing game - generate random number V2
  • 3.1.3 guessing game - read user input
  • 1.4 guessing game - realize judgment logic
  • 3.1.5 guessing game - realize the game cycle
  • 3.2 introduction to online dictionary
  • 3.2.1 Online Dictionary - packet capturing
  • 3.2.2 Online Dictionary - code generation
  • 3.2.2 Online Dictionary - code generation
  • 3.2.2 Online Dictionary - generated code interpretation
  • 3.2.3 Online Dictionary - generate request body
  • 3.2.4 Online Dictionary - parse response body
  • 3.2.5 Online Dictionary - print results
  • 3.2.6 Online Dictionary - perfect code
  • 3.3 SOCKS5 agent introduction
  • 3.3 SOCKS5 agent - principle
  • 3.3.1 SOCKS5 proxy TCP echo server
  • 3.3.2 SOCKS5 agent auth
  • 3.3.3 SOCKS5 agent - request phase
  • 3.3.4 SOCKS5 agent relay phase
  • 3.3.4 SOCKS5 agent relay phase
  • summary

This is the first note I took part in the note creation activity of "the third youth training camp - back end field"

Today is the first day of basic class

01 introduction

1.1 what is Go language

What are the characteristics of Go language?

  1. High performance and high concurrency
  2. Simple grammar and gentle learning curve
  3. Rich standard library
  4. Perfect tool chain
  5. Static link
  6. Quick compilation
  7. Cross platform
  8. garbage collection

A simple example

You can start a web service with just two sentences of code

package main

import (
 "net/http"
)

func main() {
 http.Handle("/", http.FileServer(http.Dir(".")))
 http.ListenAndServe("localhost:8080", nil)
}
copy

1.2 which companies use Go language

Which companies are using the Go language and which scenarios are it mainly used in?

  1. Byte beating has fully embraced the go language. Tens of thousands of microservices in the company are written using golang. Not long ago, the GO RPC framework KiteX was also open-source. According to the recruitment data of Lago, Tencent, Baidu, meituan, Didi, Shenxin, Ping'an, OPPO, Zhihu, qunar, 360, Jinshan, Weibo, BiliBili, qiniu PingCAP and other companies also use Go language in Daxing. Foreign companies such as Google and Facebook are also using Go language in large numbers.
  2. From the business perspective, language has been booming in cloud computing, micro services, big data, blockchain, Internet of things and other fields. Then, in cloud computing, micro services and other fields, it has a very high market share. Almost all cloud native components of Docker, Kubernetes, Istio, etcd and prometheus are implemented with Go.

1.3 why byte beating embraces GO language

  1. The original Python was changed to Go due to performance problems
  2. C++ is not suitable for online Web business
  3. Early team non Java background
  4. Good performance
  5. Simple deployment and low learning cost
  6. Promotion of internal RPC and HTTP frameworks

We know that byte beating has fully embraced the go language. In the beginning, the company's back-end business was mainly web back-end. In the early days, the team had a non Java background, and C++ was not suitable for online web business, so the first services were python. About 2014, with the growth of business volume, python encountered some performance problems.

Some teams have tried to use go for the first time and found that it is easy to get started, with high development efficiency and good performance. The development and deployment of go language is very simple, and it solves the problems brought by python

It is a headache to rely on the version of the library. After some businesses tasted the sweetness, they began to promote vigorously at the company level, and the company's internal rpc and http framework based on golang was born.

With the promotion of the framework, more and more python services are rewritten with golang. So far, golang has become the most widely used programming language in the world.

02 getting started

This chapter mainly introduces the development environment, basic syntax and standard library.

2.1 development environment - install Golang

2.1 development environment - configure the integrated development environment

2.2 basic grammar Hello World

How to configure the go development environment. Here is not a detailed introduction, just Baidu. Next, we will teach you some basic grammar of go source code through some small examples. Let's first look at the helloword in the go language. The helloworld code looks like this

package main // package main means that the file belongs to a part of the main package, which is also the entry package of the program.

import (
 "fmt" // FMT package in the standard library. This package is mainly used to input and output strings and format strings to the screen.
)

func main() {
 fmt.Println("hello world") // The main function calls fmt Println output helloword
}
copy

2.2 basic syntax - variables

The Go language variable name consists of letters, numbers and underscores. The first character cannot be a number. The general form of declaring variables is to use the var keyword:

First, specify the variable type. If the variable is not initialized, the variable defaults to zero (the value set by the system by default when the variable is not initialized).

package main
import "fmt"
func main() {
    // Declare a variable and initialize
    var a = "RUNOOB"
    fmt.Println(a)

    // Zero value without initialization
    var b int
    fmt.Println(b)

    // bool zero value is false
    var c bool
    fmt.Println(c)
}
/** output
RUNOOB
0
false
/
copy

The second method is to determine the variable type by itself according to the value.

package main
import "fmt"
func main() {
    var d = true
    fmt.Println(d)
}
// Output: true
copy

Third, if a variable has been declared with var and then declared with: =, a compilation error will occur. Format:

package main
import "fmt"
func main() {
    f := "Hane " // var f string = "Runoob"
    fmt.Println(f)
}
// Output: thousand feather
copy
package main

import (
 "fmt"
 "math"
)

func main() {

 var a = "initial"

 var b, c int = 1, 2

 var d = true

 var e float64

 f := float32(e)

 g := a + "foo"
 fmt.Println(a, b, c, d, e, f) // initial 1 2 true 0 0
 fmt.Println(g)                // initialapple

 const s string = "constant"
 const h = 500000000
 const i = 3e20 / h
 fmt.Println(s, h, i, math.Sin(h), math.Sin(i))
}
// output
// initial 1 2 true 0 0
// initialfoo                                                      
// constant 500000000 6e+11 -0.28470407323754404 0.7591753930288755
copy

2.3 basic syntax -if else

Go language provides the following conditional judgment statements:

statement

describe

if statement

An if statement consists of a Boolean expression followed by one or more statements.

If Else statement

An optional else statement can be used after the if statement. The expression in the else statement is executed when the Boolean expression is false.

if nested statement

You can embed one or more if or else if statements in an if or else if statement.

switch statement

switch statements are used to perform different actions based on different conditions.

select statement

The select statement is similar to the switch statement, but select randomly executes a runnable case. If there is no case to run, it will block until there is a case to run.

Note: Go does not have a ternary operator, so it does not support?: Conditional judgment of form.

The if else writing method in the go language is similar to that in C or c++. The difference is that there are no parentheses after if. If you write parentheses, your editor will automatically remove you when saving. The second difference is that if in Golang must be followed by braces, that is, you can't directly put the statements in if on the same line as C or c++.

package main

import "fmt"

func main() {
 // No parentheses after if
 if 7%2 == 0 {
  fmt.Println("7 is even")
 } else {
  fmt.Println("7 is odd")
 }

 if 8%4 == 0 {
  fmt.Println("8 is divisible by 4")
 }

 if num := 9; num < 0 {
  fmt.Println(num, "is negative")
 } else if num < 10 {
  fmt.Println(num, "has 1 digit")
 } else {
  fmt.Println(num, "has multiple digits")
 }
}
copy

2.4 basic syntax - loop

In go, there are no while loops and do while loops. There is only one for loop. The simplest for loop is to write nothing after for, which represents an dead loop. You can use break to jump out of the loop. In the loop, you can use break or continue to jump out or continue the loop,

The Go language provides the following types of loop processing statements:

Cycle type

describe

for loop

Repeat statement block

loop nesting

Nest one or more for loops within a for loop

Loop control statement

Loop control statements can control the execution of statements within a loop.

The GO language supports the following loop control statements:

Control statement

describe

break statement

Often used to break the current for loop or jump out of a switch statement

continue statement

Skip the remaining statements of the current loop and proceed to the next loop.

goto statement

Transfers control to the marked statement.

package main

import "fmt"

func main() {

 i := 1
 for {  // Nothing in for represents an endless loop
  fmt.Println("loop")
  break
 }
 for j := 7; j < 9; j++ {
  fmt.Println(j)
 }

 for n := 0; n < 5; n++ {
  if n%2 == 0 {
   continue
  }
  fmt.Println(n)
 }
 for i <= 3 {
  fmt.Println(i)
  i = i + 1
 }
}
copy

2.5 basic grammar switch

switch statements are used to perform different actions based on different conditions.

Switch branch structure in go language. It looks similar to C or c++. Similarly, the variable name after switch does not need parentheses.

There is a big difference here. In c++, if the switch case does not display break, it will continue to run through all cases. In the go language, it does not need to add break.

Compared with C or c++, the switch function in go language is more powerful. You can use any variable type, or even replace any if else statement. You can write conditional branches in case without adding any variables after switch. In this way, the code logic will be clearer than if else code.

package main

import (
 "fmt"
 "time"
)

func main() {

 a := 2
 switch a {
 case 1:
  fmt.Println("one")
 case 2:
  fmt.Println("two")
 case 3:
  fmt.Println("three")
 case 4, 5:
  fmt.Println("four or five")
 default:
  fmt.Println("other")
 }

 t := time.Now()
 switch {
 case t.Hour() < 12:
  fmt.Println("It's before noon")
 default:
  fmt.Println("It's after noon")
 }
}
copy
// Output:
two
It's after noon
copy

2.6 basic syntax - array

The Go language provides data structures of array types. An array is a sequence of numbered and fixed length data items with the same unique type. This type can be any original type, such as integer, string, or custom type.

An array is a sequence of elements with a number and a fixed length. For example, an array can store five int elements. For an array, you can easily get the value of a specific index or store the value of a specific index, and then you can directly print an array. However, in real business code, we rarely use arrays directly, because its length is fixed, and we use more slices.

package main

import "fmt"

func main() {

 var a [5]int
 a[4] = 100
 fmt.Println("get:", a[2])
 fmt.Println("len:", len(a))

 b := [5]int{1, 2, 3, 4, 5}
 fmt.Println(b)

 var twoD [2][3]int
 for i := 0; i < 2; i++ {
  for j := 0; j < 3; j++ {
   twoD[i][j] = i + j
  }
 }
 fmt.Println("2d: ", twoD)
}
copy
// output
get: 0
len: 5                
[1 2 3 4 5]           
2d:  [[0 1 2] [1 2 3]]
copy

2.7 basic grammar - Slicing

Go language slicing is an abstraction of arrays. The length of the go array cannot be changed, and such a set is not applicable in a specific scene. Go provides a flexible and powerful built-in type slice ("dynamic array"). Compared with the array, the length of the slice is not fixed, and elements can be added, which may increase the capacity of the slice.

We can use make to create a slice. We can take values like an array, and use append to append elements.

If you pay attention to the use of append, you must assign the result of append to the original array. Because the principle of slice is actually that it has a length and capacity stored in it, plus a pointer to an array. When you perform the append operation, if the capacity is insufficient, the capacity will be expanded and a new slice will be returned.

The length can also be specified during slice initialization. Slice has the same slicing operations as python. For example, this represents taking out the elements from the second to the fifth position, excluding the fifth element. However, unlike python, negative index is not supported here.

package main

import "fmt"

func main() {

 s := make([]string, 3)
 s[0] = "a"
 s[1] = "b"
 s[2] = "c"
 fmt.Println("get:", s[2])   // c
 fmt.Println("len:", len(s)) // 3

 s = append(s, "d")
 s = append(s, "e", "f")
 fmt.Println(s) // [a b c d e f]

 c := make([]string, len(s))
 copy(c, s)
 fmt.Println(c) // [a b c d e f]

 fmt.Println(s[2:5]) // [c d e]
 fmt.Println(s[:5])  // [a b c d e]
 fmt.Println(s[2:])  // [c d e f]

 good := []string{"g", "o", "o", "d"}
 fmt.Println(good) // [g o o d]
}
copy

2.8 basic syntax map

A Map is an unordered collection of key value pairs. The most important point of Map is to quickly retrieve data through key. Key is similar to index and points to the value of data. A Map is a collection, so we can iterate over it as we iterate over arrays and slices. However, the Map is unordered, and we cannot determine its return order, because the Map is implemented using a hash table.

Map, in other programming languages, may be called a hash or a dictionary. Map is the most frequently used data structure in actual use. We can use make to create an empty map. Here, we need two types. The first is the type of key, here is string, and the other is the type of value, here is int. You can store or retrieve key value pairs from it. You can use delete to delete key value pairs.

The map of golang is completely unordered. When traversing, it will not be output in alphabetical order or in insertion order, but in random order.

package main

import "fmt"

func main() {
 m := make(map[string]int)
 m["one"] = 1
 m["two"] = 2
 fmt.Println(m)           // map[one:1 two:2]
 fmt.Println(len(m))      // 2
 fmt.Println(m["one"])    // 1
 fmt.Println(m["unknow"]) // 0

 r, ok := m["unknow"]
 fmt.Println(r, ok) // 0 false

 delete(m, "one")

 m2 := map[string]int{"one": 1, "two": 2}
 var m3 = map[string]int{"one": 1, "two": 2}
 fmt.Println(m2, m3)
}
copy

2.9 basic grammar range

The range keyword in the Go language is used to iterate over the elements of an array, slice, channel, or set (map) in a for loop. In the array and slice, it returns the index of the element and the value corresponding to the index, and returns the key value pair in the collection.

Use range to quickly traverse, so that the code can be more concise. During range traversal, two values will be returned for the array, the first is the index, and the second is the value of the corresponding position. If we don't need an index, we can ignore it with underscores.

package main

import "fmt"

func main() {
 nums := []int{2, 3, 4}
 sum := 0
 for i, num := range nums {
  sum += num
  if num == 2 {
   fmt.Println("index:", i, "num:", num) // index: 0 num: 2
  }
 }
 fmt.Println(sum) // 9

 m := map[string]string{"a": "A", "b": "B"}
 for k, v := range m {
  fmt.Println(k, v) // b 8; a A
 }
 for k := range m {
  fmt.Println("key", k) // key a; key b
 }
}
copy

2.10 basic syntax - functions

The Go language has at least one main() function (java is called a method). You can divide different functions by functions. Logically, each function performs a specified task. Function declarations tell the compiler the name, return type, and parameters of the function.

The Go language standard library provides a variety of available built-in functions. For example, the len() function can accept different type parameters and return the length of that type. If we pass in a string, we return the length of the string. If we pass in an array, we return the number of elements contained in the array.

package main

import "fmt"

func add(a int, b int) int {
 return a + b
}

func add2(a, b int) int {
 return a + b
}

func exists(m map[string]string, k string) (v string, ok bool) {
 v, ok = m[k]
 return v, ok
}

func main() {
 res := add(1, 2)
 fmt.Println(res) // 3

 v, ok := exists(map[string]string{"a": "A"}, "a")
 fmt.Println(v, ok) // A True
}
copy

This is a simple function to add two variables in Golang. Unlike many other languages, Golang has a postpositive variable type. Functions in Golang natively support returning multiple values. In the actual business logic code, almost all functions return two values. The first value is the real return result, and the second value is an error message.

2.11 basic syntax - pointer

As we all know, a variable is a convenient placeholder for referencing a computer memory address.

The address fetching character of Go language is &. If it is used before a variable, the memory address of the corresponding variable will be returned.

package main

import "fmt"

func add2(n int) {
 n += 2
}

func add2ptr(n *int) {
 *n += 2
}

func main() {
 n := 5
 add2(n)
 fmt.Println(n) // 5
 add2ptr(&n)
 fmt.Println(n) // 7
}
copy

go also supports pointers. One of the main uses of pointers is to modify the incoming parameters. Let's look at this function. This function tries to put a variable +2. But it is invalid to write simply like the above. Because the parameter passed in to the function is actually a copy, it also means that this +2 is a +2 to that copy, which does not work. If we need to work, we need to write that type as a pointer type. In order to match the type, we will add a & symbol when calling.

2.12 basic grammar - structure

Arrays in Go language can store the same type of data, but we can define different data types for different items in the structure. A structure is a data set composed of a series of data of the same type or different types.

package main

import "fmt"

type user struct {
 name     string
 password string
}

func main() {
 a := user{name: "wang", password: "1024"}
 b := user{"wang", "1024"}
 c := user{name: "wang"}
 c.password = "1024"
 var d user
 d.name = "wang"
 d.password = "1024"

 fmt.Println(a, b, c, d)                 // {wang 1024} {wang 1024} {wang 1024} {wang 1024}
 fmt.Println(checkPassword(a, "haha"))   // false
 fmt.Println(checkPassword2(&a, "haha")) // false
}

func checkPassword(u user, password string) bool {
 return u.password == password
}

func checkPassword2(u *user, password string) bool {
 return u.password == password
}
copy

Struct is a collection of fields with types. For example, the user structure here contains two fields, name and password. We can initialize a structure variable with the name of the structure. When constructing, we need to pass in the initial value of each field. You can also use this key value pair to specify the initial value, so that only a part of the fields can be initialized. For the same structure, we can also support pointers, so that we can modify the structure and avoid the copying cost of some large structures in some cases.

2.13 basic grammar - structure method

In Golang, you can define some methods for structures. It will be a little similar to class member functions in other languages. As shown here, we changed the implementation of checkPassword in the above example from a common function to a structure method. In this way, the user can call a.checkPassword("xx"). The specific code modification is to write the first parameter in parentheses in front of the function name.

There are also two ways to implement the method of structure, one with pointer and the other without pointer. The difference between them is that if you take a pointer, you can modify the structure. If you do not take a pointer, you are actually operating on a copy, and you cannot modify the structure.

package main

import "fmt"

type user struct {
 name     string
 password string
}

func (u user) checkPassword(password string) bool {
 return u.password == password
}

func (u *user) resetPassword(password string) {
 u.password = password
}

func main() {
 a := user{name: "wang", password: "1024"}
 a.resetPassword("2048")
 fmt.Println(a.checkPassword("2048")) // true
}
copy

2.14 basic syntax - error handling

The Go language provides a very simple error handling mechanism through the built-in error interface.

error type is an interface type. This is its definition:

type error interface {
    Error() string
}
copy

We can generate error messages by implementing the error interface type in the code.

Function usually returns an error message in the last return value. Use errors New returns an error message:

func Sqrt(f float64) (float64, error) {
    if f < 0 {
        return 0, errors.New("math: square root of negative number")
    }
    // achieve
}
copy

In the following example, we pass a negative number when calling Sqrt, and then we get the non nil error object. Comparing this object with nil, the result is true, so fmt Println (the FMT package will call the error method when processing errors) is called to output errors. Please see the following example code:

result, err:= Sqrt(-1)

if err != nil {
   fmt.Println(err)
}
copy

Error handling in the go language conforms to the language habit of using a separate return value to pass error information.

It is different from the exceptions used by Java itself. The processing method of go language can clearly know which function returned an error, and can use a simple if else to handle the error. In a function, we can add an error to the return value type of that function, which means that the function may return an error.

When implementing a function, return needs to return two values at the same time. Or if there is an error, you can return nil and an error. If not, return the original result and nil

package main

import (
 "errors"
 "fmt"
)

type user struct {
 name     string
 password string
}

func findUser(users []user, name string) (v *user, err error) {
 for _, u := range users {
  if u.name == name {
   return &u, nil
  }
 }
 return nil, errors.New("not found")
}

func main() {
 u, err := findUser([]user{{"wang", "1024"}}, "wang")
 if err != nil {
  fmt.Println(err)
  return
 }
 fmt.Println(u.name) // wang

 if u, err := findUser([]user{{"wang", "1024"}}, "li"); err != nil {
  fmt.Println(err) // not found
  return
 } else {
  fmt.Println(u.name)
 }
}
copy

2.15 basic syntax - string operation

Let's take a look at the string operations in the go language. There are many commonly used string tool functions in the string package of the standard library. For example, contains determines whether a string contains another string, count s the number of character audits, and index finds the location of a character audit. join joins multiple strings

package main

import (
 "fmt"
 "strings"
)

func main() {
 a := "hello"
 fmt.Println(strings.Contains(a, "ll"))                // true
 fmt.Println(strings.Count(a, "l"))                    // 2 Statistics
 fmt.Println(strings.HasPrefix(a, "he"))               // true
 fmt.Println(strings.HasSuffix(a, "llo"))              // true
 fmt.Println(strings.Index(a, "ll"))                   // 2
 fmt.Println(strings.Join([]string{"he", "llo"}, "-")) // He LLO splicing
 fmt.Println(strings.Repeat(a, 2))                     // hellohello  
 fmt.Println(strings.Replace(a, "e", "E", -1))         // hEllo  
 fmt.Println(strings.Split("a-b-c", "-"))              // [a b c] split
 fmt.Println(strings.ToLower(a))                       // hello to lowercase
 fmt.Println(strings.ToUpper(a))                       // HELLO to capital
 fmt.Println(len(a))                                   // 5 length
 b := "hello"
 fmt.Println(len(b)) // 6
}
copy

2.16 basic syntax - string formatting

There are many string format related methods in the FMT package of the standard library, such as printf, which is similar to the printf function in C language. The difference is that in the go language, you can easily use%v to print variables of any type without distinguishing the fractional string. You can also print detailed results with%+v,%\v is more detailed.

package main

import "fmt"

type point struct {
 x, y int
}

func main() {
 s := "hello"
 n := 123
 p := point{1, 2}
 fmt.Println(s, n) // hello 123
 fmt.Println(p)    // {1 2}

 fmt.Printf("s=%v\n", s)  // s=hello
 fmt.Printf("n=%v\n", n)  // n=123
 fmt.Printf("p=%v\n", p)  // p={1 2}
 fmt.Printf("p=%+v\n", p) // p={x:1 y:2}
 fmt.Printf("p=%#v\n", p) // p=main.point{x:1, y:2}

 f := 3.141592653
 fmt.Println(f)          // 3.141592653
 fmt.Printf("%.2f\n", f) // 3.14
}

copy

2.17 basic syntax JSON processing

Let's take a look at JSON operations. JSON operations in the go language are very simple. For an existing structure, we can do nothing, as long as we ensure that the first letter of each field is capitalized, that is, it is a public field. Then this structure can use json Marshaler deserializes it into a JSON string.

The serialized string can also use json Unmarshal de serializes to an empty variable.

If the string is serialized by default, its style starts with an uppercase letter, not an underscore. We can use json tag and other syntax to modify the field name in the output JSON result later.

package main

import (
 "encoding/json"
 "fmt"
)

type userInfo struct {
 Name  string
 Age   int `json:"age"`
 Hobby []string
}

func main() {
 a := userInfo{Name: "wang", Age: 18, Hobby: []string{"Golang", "TypeScript"}}
 buf, err := json.Marshal(a)
 if err != nil {
  panic(err)
 }
 fmt.Println(buf)         // [123 34 78 97...]
 fmt.Println(string(buf)) // {"Name":"wang","age":18,"Hobby":["Golang","TypeScript"]}

 buf, err = json.MarshalIndent(a, "", "\t")
 if err != nil {
  panic(err)
 }
 fmt.Println(string(buf))

 var b userInfo
 err = json.Unmarshal(buf, &b)
 if err != nil {
  panic(err)
 }
 fmt.Printf("%#v\n", b) // main.userInfo{Name:"wang", Age:18, Hobby:[]string{"Golang", "TypeScript"}}
}
copy

2.18 basic grammar - time processing

The most commonly used in go language is time Now() to get the current time, and then you can also use time Date() to construct a time with time zone, and the constructed time. There are many methods to obtain the year, day, hour, minute and second of this time point, and then you can use sub to subtract the two times. When interacting with some systems, we often use time stamps. Yes UNIX to get the timestamp time Format(), time Parse()

package main

import (
 "fmt"
 "time"
)

func main() {
 now := time.Now()
 fmt.Println(now) // 2022-03-27 18:04:59.433297 +0800 CST m=+0.000087933
 t := time.Date(2022, 3, 27, 1, 25, 36, 0, time.UTC)
 t2 := time.Date(2022, 3, 27, 2, 30, 36, 0, time.UTC)
 fmt.Println(t)                                                  // 2022-03-27 01:25:36 +0000 UTC
 fmt.Println(t.Year(), t.Month(), t.Day(), t.Hour(), t.Minute()) // 2022 March 27 1 25
 fmt.Println(t.Format("2006-01-02 15:04:05"))                    // 2022-03-27 01:25:36
 diff := t2.Sub(t)
 fmt.Println(diff)                           // 1h5m0s
 fmt.Println(diff.Minutes(), diff.Seconds()) // 65 3900
 t3, err := time.Parse("2006-01-02 15:04:05", "2022-03-27 01:25:36")
 if err != nil {
  panic(err)
 }
 fmt.Println(t3 == t)    // true
 fmt.Println(now.Unix()) // 1648738080
}
copy

2.19 basic grammar - number parsing

Conversion between string and number. In the go language, the conversion between string and number types is under the strconv package, which is the abbreviation of string convert. You can use strconv Parseint() or strconv Parsefloat() to parse a string. You can use strconv ATOI ("AAA") converts a decimal string to a number. You can use strconv Itoa () converts a number to a string. If the input is illegal, these functions will return error

package main

import (
 "fmt"
 "strconv"
)

func main() {
 f, _ := strconv.ParseFloat("1.234", 64)
 fmt.Println(f) // 1.234

 n, _ := strconv.ParseInt("111", 10, 64)
 fmt.Println(n) // 111

 n, _ = strconv.ParseInt("0x1000", 0, 64)
 fmt.Println(n) // 4096

 n2, _ := strconv.Atoi("123")
 fmt.Println(n2) // 123
 n2, err := strconv.Atoi("AAA")
 fmt.Println(n2, err) // 0 strconv.Atoi: parsing "AAA": invalid syntax
 n3 := strconv.Itoa(123)
 fmt.Println(n3) // 123

}
copy

2.20 basic syntax - process information

In go, we can use os Args to get the specified command line parameters during program execution. For example, we compile a binary file, command. Followed by abcd to start, the output is os Args will be a slice with a length of 5, and the first member represents the name of the binary itself. We can use os Getenv ("path") to read environment variables.

package main

import (
 "fmt"
 "os"
 "os/exec"
)

func main() {
 // go run example/20-env/main.go a b c d
 fmt.Println(os.Args) // [/var/folders/8p/n34xxfnx38dg8bv_x8l62t_m0000gn/T/go-build3406981276/b001/exe/main a b c d]
 fmt.Println("-----------")
 fmt.Println(os.Getenv("PATH")) // /usr/local/go/bin...
 fmt.Println("-----------")
 fmt.Println(os.Setenv("AA", "BB"))
 fmt.Println("-----------")

 buf, err := exec.Command("grep", "127.0.0.1", "/etc/hosts").CombinedOutput()
 if err != nil {
  panic(err)
 }
 fmt.Println(string(buf)) // 127.0.0.1       localhost
}
copy

03 actual combat

Here we mainly introduce guessing games, online dictionaries and SOCKS5 agents.

3.1 introduction to guessing game

Here we use Golang to build a number guessing game. In this game, the program will first generate a random integer between 1 and 100, and then prompt the player to guess. Each time the player enters a number, the program will tell the player whether the guessed value is higher or lower than the secret value. If you guessed right, tell the player to win and exit the program.

3.1.1 guessing game - generate random numbers

package main

import (
 "fmt"
 "math/rand"
)

func main() {
 maxNum := 100
 secretNumber := rand.Intn(maxNum)
 fmt.Println("The secret number is ", secretNumber)
}
copy

When the program runs, it will generate a random number between 0 and 100. Let's start by generating this random number. To generate random numbers, we need to use the math/rand package. The code of our first version is like this. First, we import the fmt package and math/rand package, and define a variable. maxNum is 100. Use rand INTN to generate a random number, and then print out the random number.

3.1.2 guessing game - generate random number effect

We found that the same number was printed on the screen every time. This is not what we want. Why?

3.1.2 guessing game - generate random number V2

We use time Now() Unixnano() to initialize the random seed.

3.1.3 guessing game - read user input

Achieve user input and output, and into digital. We can use its ReadString method to read a row. If it fails, we will print the error and exit. The result returned by ReadString contains the end newline character. We remove it and convert it to a number. If the conversion fails, we will also print an error and exit.

1.4 guessing game - realize judgment logic

Now we have a secret value, and then we read a value from the user's input. Let's compare the size of the two values. If the value entered by the user is larger than the secret value, tell the user that the value you guessed is too large. Please try again. If it is small, it is the same. If it is equal, we will tell the user that you have won.

3.1.5 guessing game - realize the game cycle

At this point, our program can work normally, but the player can only enter a guess. Whether the guess is correct or not, the program will exit abruptly. In order to change this behavior and make the game play normally, we need to add a loop. We move the code just now into a for loop, and then change the return to continue so that we can continue the loop in case of errors. break when the user input is correct, so as to exit the game when the user wins.

In this way, we have successfully built a guessing game in Golang. In this process, we reviewed many previous concepts, such as variable loop, function control flow, and error handling.

3.2 introduction to online dictionary

Implement a command line typesetting dictionary

Users can query a word on the command line. We can call the third-party API to query the word translation and print it out. In this example, we will learn how to use go language to send HTTP requests and parse json. We will also learn how to use code generation to improve development efficiency.

3.2.1 Online Dictionary - packet capturing

Let's first take a look at the API we need to use, taking the online translation provided by Caiyun technology as an example. Please open the color cloud translation web page first, and then right-click F12 to check the developer tool of the browser.

Translated by Caiyun: https://fanyi.caiyunapp.com/#/

At this point, we click the translate button, and the browser will send a series of requests. We can easily find the request used to query words.

This is an HTTP post request. The header of the request is quite complex. There are about a dozen. Then the request header is a json with two fields. One represents the language you want to convert from to. The source is the word you want to query. There will be Wiki and dictionary fields in the returned result of API. The results we need to use are mainly in the dictionary Explanations field. Other fields also include phonetic symbols and other information.

3.2.2 Online Dictionary - code generation

We need to send this request in Golang. Because the request is complex, it is troublesome to construct code. In fact, we have a very simple way to generate code. We can right-click copy as cur in the browser. After copy, you can paste the curl command on the terminal. It should succeed

3.2.2 Online Dictionary - code generation

Then we open a website https://curlconverter.com/#go , paste the curl request, and select Golang in the language on the right to see a long string of code. We directly copy it into our editor. Several header s are complex. The generated code has compilation errors caused by escape. Just delete these lines.

3.2.2 Online Dictionary - generated code interpretation

package main

import (
 "fmt"
 "io/ioutil"
 "log"
 "net/http"
 "strings"
)

func main() {
 client := &http.Client{}
 var data = strings.NewReader(`{"trans_type":"en2zh","source":"good"}`)
 req, err := http.NewRequest("POST", "https://api.interpreter.caiyunai.com/v1/dict", data)
 if err != nil {
  log.Fatal(err)
 }
 req.Header.Set("Accept", "application/json, text/plain, */*")
 req.Header.Set("Accept-Language", "zh-CN,zh;q=0.9")
 req.Header.Set("Connection", "keep-alive")
 req.Header.Set("Content-Type", "application/json;charset=UTF-8")
 req.Header.Set("Origin", "https://fanyi.caiyunapp.com")
 req.Header.Set("Referer", "https://fanyi.caiyunapp.com/")
 req.Header.Set("Sec-Fetch-Dest", "empty")
 req.Header.Set("Sec-Fetch-Mode", "cors")
 req.Header.Set("Sec-Fetch-Site", "cross-site")
 req.Header.Set("User-Agent", "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/101.0.4951.54 Safari/537.36")
 req.Header.Set("X-Authorization", "token:qgemv4jr1y38jyq6vhvi")
 req.Header.Set("app-name", "xy")
 req.Header.Set("os-type", "web")
 req.Header.Set("sec-ch-ua", `" Not A;Brand";v="99", "Chromium";v="101", "Google Chrome";v="101"`)
 req.Header.Set("sec-ch-ua-mobile", "?0")
 req.Header.Set("sec-ch-ua-platform", `"Windows"`)
 resp, err := client.Do(req)
 if err != nil {
  log.Fatal(err)
 }
 defer resp.Body.Close()
 bodyText, err := ioutil.ReadAll(resp.Body)
 if err != nil {
  log.Fatal(err)
 }
 fmt.Printf("%s\n", bodyText)
}
copy

Let's take a look at the generated code:

  1. First, in line 12, we create an HTTP client. Many parameters can be specified during creation, including whether to use cookie s for request timeout. The next step is to construct an HTTP request, which is a post request.
  2. Then http will be used Newrequest: the first parameter is http method POST, the second parameter is URL, and the last parameter is body. Body is a read-only stream because it may be very large to support streaming. We used strings Newreader to convert a string into a stream. In this way, we have successfully constructed an HTTP request. Next, we need to set a stack of header s for this HTTP request
  3. Next, we call client Do request to get the response. If the request fails, the error will return non nil, print the error and exit the process. Response has its HTTP status code, response header and body. Body is also a stream. In golang, to avoid resource leakage, you need to add a defer to manually close the stream. This defer will be executed after the function is run. Next we use ioutil Readll to read the stream and get the whole body. We print it out with print.

When we run the generated code, we can see that we have successfully issued the request and printed the returned JSON. But now the input is fixed. We need to input from a variable. We need to use JSON serialization

3.2.3 Online Dictionary - generate request body

3.2.4 Online Dictionary - parse response body

Next, we need to parse the response body. In js/Python scripting languages, body is a dictionary or map structure from which values can be obtained directly. But golang is a strongly typed language, which is not a best practice. The more common way is to write a structure like the request, and deserialize the returned JSON into the structure. However, we can see in the browser that the structure returned by this API is very complex. It is very tedious and error prone to define the structure field.

Here's a trick: there is a corresponding code generation tool on the Internet. We can open this website https://oktools.net/json2go , paste the JSON string, so that we can generate the corresponding structure. At some time, if we do not need to do many fine operations on the returned result, we can choose to convert nesting, which can make the generated code more compact.

So we get a response structure. Next, we modify the code. First, we define an object of the response structure, and then we use json Unmarshal deserializes the body into this structure, and then tries to print it out

Now let's run it again. When printing here,%\v is used to make the printed results easier to read. Now that we are very close to the final version, we need to modify the code to print the specific fields in the response.

3.2.5 Online Dictionary - print results

Looking at the json, we can see that the result we need is in the dictionary Explanations We use the for range loop to iterate over it, and then directly print the structure. Referring to the display mode of some dictionaries, we can print the word and its phonetic symbols in front of it. There are English phonetic symbols and American phonetic symbols.

3.2.6 Online Dictionary - perfect code

Now the input of our program is still dead. We change the main body of the code into a query function, and the query words are passed in as parameters. Then we write a simple main function. This main function first determines the number of commands and parameters. If it is not two, then we print out the error message and exit the program. Otherwise, get the word entered by the user, and then execute the query function.

package main

import (
 "bytes"
 "encoding/json"
 "fmt"
 "io/ioutil"
 "log"
 "net/http"
 "os"
)

type DictRequest struct {
 TransType string `json:"trans_type"`
 Source    string `json:"source"`
 UserID    string `json:"user_id"`
}

type DictResponse struct {
 Rc   int `json:"rc"`
 Wiki struct {
  KnownInLaguages int `json:"known_in_laguages"`
  Description     struct {
   Source string      `json:"source"`
   Target interface{} `json:"target"`
  } `json:"description"`
  ID   string `json:"id"`
  Item struct {
   Source string `json:"source"`
   Target string `json:"target"`
  } `json:"item"`
  ImageURL  string `json:"image_url"`
  IsSubject string `json:"is_subject"`
  Sitelink  string `json:"sitelink"`
 } `json:"wiki"`
 Dictionary struct {
  Prons struct {
   EnUs string `json:"en-us"`
   En   string `json:"en"`
  } `json:"prons"`
  Explanations []string      `json:"explanations"`
  Synonym      []string      `json:"synonym"`
  Antonym      []string      `json:"antonym"`
  WqxExample   [][]string    `json:"wqx_example"`
  Entry        string        `json:"entry"`
  Type         string        `json:"type"`
  Related      []interface{} `json:"related"`
  Source       string        `json:"source"`
 } `json:"dictionary"`
}

func query(word string) {
 client := &http.Client{}
 request := DictRequest{TransType: "en2zh", Source: word}
 buf, err := json.Marshal(request)
 if err != nil {
  log.Fatal(err)
 }
 var data = bytes.NewReader(buf)
 req, err := http.NewRequest("POST", "https://api.interpreter.caiyunai.com/v1/dict", data)
 if err != nil {
  log.Fatal(err)
 }
 req.Header.Set("Connection", "keep-alive")
 req.Header.Set("DNT", "1")
 req.Header.Set("os-version", "")
 req.Header.Set("sec-ch-ua-mobile", "?0")
 req.Header.Set("User-Agent", "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/99.0.4844.51 Safari/537.36")
 req.Header.Set("app-name", "xy")
 req.Header.Set("Content-Type", "application/json;charset=UTF-8")
 req.Header.Set("Accept", "application/json, text/plain, */*")
 req.Header.Set("device-id", "")
 req.Header.Set("os-type", "web")
 req.Header.Set("X-Authorization", "token:qgemv4jr1y38jyq6vhvi")
 req.Header.Set("Origin", "https://fanyi.caiyunapp.com")
 req.Header.Set("Sec-Fetch-Site", "cross-site")
 req.Header.Set("Sec-Fetch-Mode", "cors")
 req.Header.Set("Sec-Fetch-Dest", "empty")
 req.Header.Set("Referer", "https://fanyi.caiyunapp.com/")
 req.Header.Set("Accept-Language", "zh-CN,zh;q=0.9")
 req.Header.Set("Cookie", "_ym_uid=16456948721020430059; _ym_d=1645694872")
 resp, err := client.Do(req)
 if err != nil {
  log.Fatal(err)
 }
 defer resp.Body.Close()
 bodyText, err := ioutil.ReadAll(resp.Body)
 if err != nil {
  log.Fatal(err)
 }
 if resp.StatusCode != 200 {
  log.Fatal("bad StatusCode:", resp.StatusCode, "body", string(bodyText))
 }
 var dictResponse DictResponse
 err = json.Unmarshal(bodyText, &dictResponse)
 if err != nil {
  log.Fatal(err)
 }
 fmt.Println(word, "UK:", dictResponse.Dictionary.Prons.En, "US:", dictResponse.Dictionary.Prons.EnUs)
 for _, item := range dictResponse.Dictionary.Explanations {
  fmt.Println(item)
 }
}

func main() {
 if len(os.Args) != 2 {
  fmt.Fprintf(os.Stderr, `usage: simpleDict WORD example: simpleDict hello`)
  os.Exit(1)
 }
 word := os.Args[1]
 query(word)
}
copy

In this way, even if our command-line dictionary is completed, we can simply try it.

3.3 SOCKS5 agent introduction

Let's write a socks5 proxy server. When we talk about proxy servers, the first thing we think of is to climb over the wall. Unfortunately, although socks5 protocol is a proxy protocol, it can not be used to cross the wall. Its protocols are plaintext transmission.

This protocol has a long history and was born in the early days of the Internet. Its purpose is that, for example, some enterprises have strict firewall policies in order to ensure the security of their intranet, but the side effect is that it will be troublesome to access some resources.

socks5 is equivalent to opening a hole in the firewall so that authorized users can access all internal resources through a single port. In fact, many wall climbing software eventually exposes a socks5 protocol port. If a student has developed a crawler, he knows that it is easy to encounter IP access frequency exceeding the limit in the process of crawling. At this time, many people will go to the Internet to find some proxy IP pools. The protocol of many agents in these proxy IP pools is socks5.

go-by-example\proxy\v4> go run .\main.go curl --socks5 127.0.0.1:1080 -v http://www.qq.com

3.3 SOCKS5 agent - principle

Next, let's take a look at the working principle of socks5 protocol. If a normal browser accesses a website without passing through a proxy server, it first establishes a TCP connection with the other website, then shakes hands three times. After shaking hands, it initiates an HTTP request, and then the service returns an HTTP response. If the proxy server is set up, the process will become more complicated

  1. First, the browser establishes a TCP connection with the socks5 proxy, and then the proxy establishes a TCP connection with the real server. It can be divided into four stages: handshake stage, authentication stage, request stage and relay stage.
  2. In the first handshake phase, the browser will send a request to the socks5 agent. The package includes a protocol version number and the types of authentication supported. The socks5 server will select an authentication method and return it to the browser. If 00 is returned, it means that authentication is not required. If other types are returned, the authentication process will start. We will not outline the authentication process here.
  3. The third stage is the request stage. After authentication, the browser will send a request to the socks5 server. The main information includes the version number and the type of request. Generally, the connection request means that the proxy server wants to establish a TCP connection with a domain name or an IP address or a port. After receiving the response, the proxy server will actually establish a connection with the back-end server, and then return a response
  4. The fourth stage is the relay stage. At this time, the browser will send a normal sending request, and then the proxy server will directly convert the request to the real server after receiving the request. Then, if the real server returns a response later, it will also forward the request to the browser. In fact, the proxy server does not care about the details of traffic. It can be HTTP traffic or other TCP meteors. This is the working principle of socks5 protocol. Next, we will try to implement it simply.

3.3.1 SOCKS5 proxy TCP echo server

First, we write a simple TCP echo server in go. For the convenience of testing, the working logic of the server is very simple. The server will reply to whatever you send. The code will look like this:

  1. First, we use net Listen listens to a port and returns a server. In an endless loop, each time you accept a request, a connection will be returned. Next, we will handle the connection in a process function. Note that there will be a go keyword in front of this. This represents starting a goroutinue, which can be temporarily compared to starting a sub thread in other languages. But the overhead of goroutinue here is much smaller than that of sub threads, and it can easily handle tens of thousands of concurrency.
  2. Next is the implementation of the process function. The first step is to add a defer connection Close(), defer is a syntax in Golang. The meaning of this line is to close the connection when the function exits, otherwise there will be resource leakage.
  3. Next, we will use bufio Newreader is used to create a buffered read-only stream, which is also useful in the previous guessing game. The function of buffered stream is to reduce the number of underlying system calls. For example, in order to facilitate byte by byte reading, the underlying system may be combined into several large reading operations. And the buffered stream will have more tools to read data. We can simply call that readbyte function to read a single byte. Then write this byte into the connection.
package main

import (
 "bufio"
 "log"
 "net"
)

func main() {
 server, err := net.Listen("tcp", "127.0.0.1:1080")
 if err != nil {
  panic(err)
 }
 for {
  client, err := server.Accept()
  if err != nil {
   log.Printf("Accept failed %v", err)
   continue
  }
  go process(client)
 }
}

func process(conn net.Conn) {
 defer conn.Close()
 reader := bufio.NewReader(conn)
 for {
  b, err := reader.ReadByte()
  if err != nil {
   break
  }
  _, err = conn.Write([]byte{b})
  if err != nil {
   break
  }
 }
}
copy

Let's simply test our first TCP server, and then the test will need to use the nc command (the window needs to be installed). We use nc 127.0.0.1 1080, enter Hello, and the server will return hello to you.

3.3.2 SOCKS5 agent auth

package main

import (
 "bufio"
 "fmt"
 "io"
 "log"
 "net"
)

const socks5Ver = 0x05
const cmdBind = 0x01
const atypIPV4 = 0x01
const atypeHOST = 0x03
const atypeIPV6 = 0x04

func main() {
 server, err := net.Listen("tcp", "127.0.0.1:1080")
 if err != nil {
  panic(err)
 }
 for {
  client, err := server.Accept()
  if err != nil {
   log.Printf("Accept failed %v", err)
   continue
  }
  go process(client)
 }
}

func process(conn net.Conn) {
 defer conn.Close()
 reader := bufio.NewReader(conn)
 err := auth(reader, conn)
 if err != nil {
  log.Printf("client %v auth failed:%v", conn.RemoteAddr(), err)
  return
 }
 log.Println("auth success")
}

func auth(reader *bufio.Reader, conn net.Conn) (err error) {
 // +----+----------+----------+
 // |VER | NMETHODS | METHODS  |
 // +----+----------+----------+
 // | 1  |    1     | 1 to 255 |
 // +----+----------+----------+
 // VER: protocol version, socks5 is 0x05
 // NMETHODS: number of methods supporting authentication
 // METHODS: corresponds to NMETHODS. The number of bytes in METHODS depends on the value of NMETHODS. RFC predefines the meanings of some values as follows:
 // X'00' NO AUTHENTICATION REQUIRED
 // X'02' USERNAME/PASSWORD

 ver, err := reader.ReadByte()
 if err != nil {
  return fmt.Errorf("read ver failed:%w", err)
 }
 if ver != socks5Ver {
  return fmt.Errorf("not supported ver:%v", ver)
 }
 methodSize, err := reader.ReadByte()
 if err != nil {
  return fmt.Errorf("read methodSize failed:%w", err)
 }
 method := make([]byte, methodSize)
 _, err = io.ReadFull(reader, method)
 if err != nil {
  return fmt.Errorf("read method failed:%w", err)
 }
 log.Println("ver", ver, "method", method)
 // +----+--------+
 // |VER | METHOD |
 // +----+--------+
 // | 1  |   1    |
 // +----+--------+
 _, err = conn.Write([]byte{socks5Ver, 0x00})
 if err != nil {
  return fmt.Errorf("write failed:%w", err)
 }
 return nil
}
copy

In this way, we have completed a TCP server that can return your input information. Next, we will start the first step of implementing the protocol, the authentication phase, which will become more complex from this part. We implement an empty auth function, call it in the process function, and then write the code of the auth function. Let's recall the logic of the authentication phase,

In the first step, the browser will send a package to the proxy server. Then the package has three fields. The first field, version, that is, the protocol version number, is fixed at 5. The second field, methods, is the number of authenticated methods. The third field is the code of each method. 0 means no authentication is required, and 2 means user name and password authentication

First, we use read bytes to read the version number. Then, if the version number is not socket 5, an error is returned directly. Next, we read the method size, which is also a byte. Then we need to make a slice of corresponding length, using io Readfull fills it in. Here, we will print the version number and authentication method obtained. At this time, the proxy server also needs to return a response. The return package includes two fields, one is version and the other is method, that is, the selected authentication method. Currently, we are only going to implement the method that does not need authentication, that is, 00. Let's test the effect of the current version with the curl command

3.3.3 SOCKS5 agent - request phase

Next, we start to do the third step. In the request phase, we try to read the package carrying the URL or IP address + port, and then print it out. We implement a connect function similar to auth function, which is also called in process. Then the code that implements the connect function.

Let's recall the logic of the request phase. The browser will send a package, which contains the following six fields: version version number or 5. command represents the type of request. We only support connection requests, that is, let the proxy service establish a new TCP connection. RSV reserves fields and ignores them. Atype is the target address type, which may be IPV 4, IPV 6, or addr under the domain name. The length of this address is the port number that varies according to the type of atype. It is two bytes. We need to read these fields one by one. The four fields of the face have a total of four bytes, which we can read out at one time. We set a buffer with a length of 4 and read it full. When it is full,

Then the 0, 1 and 3 are version cmd and type respectively. Version needs to be determined as socket5 and cmd needs to be determined as 1. The following types may be ipv4, ipv6, or host. If IPV 4 is used, we will read the buffer again, because the buffer length is just 4 bytes, and then print it into the format of IP address byte by byte and save it to the addr variable.

If it is a host, you need to read its length first, and then make a buf of corresponding length to fill it. Then convert it into a string and save it to the addr variable. IPV 6 is rarely used, so we don't support it for the time being. The last two bytes are port. We read it and convert it into numbers according to the protocol's large byte order. Since the above buffer will no longer be used by other variables, we can directly reuse the previous memory and create a temporary slice with a length of 2 for reading. In this way, we can read only two bytes at most. Next, we print out the address and port for debugging. After receiving the request package from the browser, we need to return a package. This package has many fields, but most of them will not be used.

The first one is the version number or socket5. The second is the returned type. Here, 0 is returned if successful. The third is to fill in 0 for the reserved field. The fourth is to fill in 1 for the atype address type. The fifth is to fill in 0 for the sixth is temporarily unavailable. There are 4+4+2 bytes in total, and the last 6 bytes are filled with 0.

package main

import (
 "bufio"
 "encoding/binary"
 "errors"
 "fmt"
 "io"
 "log"
 "net"
)

const socks5Ver = 0x05
const cmdBind = 0x01
const atypIPV4 = 0x01
const atypeHOST = 0x03
const atypeIPV6 = 0x04

func main() {
 server, err := net.Listen("tcp", "127.0.0.1:1080")
 if err != nil {
  panic(err)
 }
 for {
  client, err := server.Accept()
  if err != nil {
   log.Printf("Accept failed %v", err)
   continue
  }
  go process(client)
 }
}

func process(conn net.Conn) {
 defer conn.Close()
 reader := bufio.NewReader(conn)
 err := auth(reader, conn)
 if err != nil {
  log.Printf("client %v auth failed:%v", conn.RemoteAddr(), err)
  return
 }
 err = connect(reader, conn)
 if err != nil {
  log.Printf("client %v auth failed:%v", conn.RemoteAddr(), err)
  return
 }
}

func auth(reader *bufio.Reader, conn net.Conn) (err error) {
 // +----+----------+----------+
 // |VER | NMETHODS | METHODS  |
 // +----+----------+----------+
 // | 1  |    1     | 1 to 255 |
 // +----+----------+----------+
 // VER: protocol version, socks5 is 0x05
 // NMETHODS: number of methods supporting authentication
 // METHODS: corresponds to NMETHODS. The number of bytes in METHODS depends on the value of NMETHODS. RFC predefines the meanings of some values as follows:
 // X'00' NO AUTHENTICATION REQUIRED
 // X'02' USERNAME/PASSWORD

 ver, err := reader.ReadByte()
 if err != nil {
  return fmt.Errorf("read ver failed:%w", err)
 }
 if ver != socks5Ver {
  return fmt.Errorf("not supported ver:%v", ver)
 }
 methodSize, err := reader.ReadByte()
 if err != nil {
  return fmt.Errorf("read methodSize failed:%w", err)
 }
 method := make([]byte, methodSize)
 _, err = io.ReadFull(reader, method)
 if err != nil {
  return fmt.Errorf("read method failed:%w", err)
 }

 // +----+--------+
 // |VER | METHOD |
 // +----+--------+
 // | 1  |   1    |
 // +----+--------+
 _, err = conn.Write([]byte{socks5Ver, 0x00})
 if err != nil {
  return fmt.Errorf("write failed:%w", err)
 }
 return nil
}

func connect(reader *bufio.Reader, conn net.Conn) (err error) {
 // +----+-----+-------+------+----------+----------+
 // |VER | CMD |  RSV  | ATYP | DST.ADDR | DST.PORT |
 // +----+-----+-------+------+----------+----------+
 // | 1  |  1  | X'00' |  1   | Variable |    2     |
 // +----+-----+-------+------+----------+----------+
 // VER version number, and the value of socks5 is 0x05
 // CMD 0x01 indicates CONNECT request
 // RSV reserved field, value 0x00
 // ATYP destination address type, dst The data of addr corresponds to the type of this field.
 //   0x01 indicates IPv4 address, dst Addr is 4 bytes
 //   0x03 indicates domain name, dst Addr is a variable length domain name
 // Dst Addr a variable length value
 // Dst Port target port, fixed 2 bytes

 buf := make([]byte, 4)
 _, err = io.ReadFull(reader, buf)
 if err != nil {
  return fmt.Errorf("read header failed:%w", err)
 }
 ver, cmd, atyp := buf[0], buf[1], buf[3]
 if ver != socks5Ver {
  return fmt.Errorf("not supported ver:%v", ver)
 }
 if cmd != cmdBind {
  return fmt.Errorf("not supported cmd:%v", ver)
 }
 addr := ""
 switch atyp {
 case atypIPV4:
  _, err = io.ReadFull(reader, buf)
  if err != nil {
   return fmt.Errorf("read atyp failed:%w", err)
  }
  addr = fmt.Sprintf("%d.%d.%d.%d", buf[0], buf[1], buf[2], buf[3])
 case atypeHOST:
  hostSize, err := reader.ReadByte()
  if err != nil {
   return fmt.Errorf("read hostSize failed:%w", err)
  }
  host := make([]byte, hostSize)
  _, err = io.ReadFull(reader, host)
  if err != nil {
   return fmt.Errorf("read host failed:%w", err)
  }
  addr = string(host)
 case atypeIPV6:
  return errors.New("IPv6: no supported yet")
 default:
  return errors.New("invalid atyp")
 }
 _, err = io.ReadFull(reader, buf[:2])
 if err != nil {
  return fmt.Errorf("read port failed:%w", err)
 }
 port := binary.BigEndian.Uint16(buf[:2])

 log.Println("dial", addr, port)

 // +----+-----+-------+------+----------+----------+
 // |VER | REP |  RSV  | ATYP | BND.ADDR | BND.PORT |
 // +----+-----+-------+------+----------+----------+
 // | 1  |  1  | X'00' |  1   | Variable |    2     |
 // +----+-----+-------+------+----------+----------+
 // VER socks version, 0x05 here
 // REP Relay field, the content value is as follows: X '00' succeeded
 // RSV reserved fields
 // ATYPE address type
 // Bnd Address of addr service binding
 // Bnd Port bound by port service dst Port
 _, err = conn.Write([]byte{0x05, 0x00, 0x00, 0x01, 0, 0, 0, 0, 0, 0})
 if err != nil {
  return fmt.Errorf("write failed: %w", err)
 }
 return nil
}
copy

3.3.4 SOCKS5 agent relay phase

We use net Dial establish a TCP connection

After the connection is established, we also need to add a defer to close the connection. Next, you need to establish two-way data forwarding between the browser and the downstream server. Standard library io Copy can realize one-way data forwarding. For two-way forwarding, you need to start two goroutinue s.

 _, err = io.ReadFull(reader, buf[:2])
 if err != nil {
  return fmt.Errorf("read port failed:%w", err)
 }
 port := binary.BigEndian.Uint16(buf[:2])

 dest, err := net.Dial("tcp", fmt.Sprintf("%v:%v", addr, port))
 if err != nil {
  return fmt.Errorf("dial dst failed:%w", err)
 }
 defer dest.Close()
 log.Println("dial", addr, port)
copy

3.3.4 SOCKS5 agent relay phase

We can try to test again in the browser. To test the proxy in the browser, we need to install the SwitchyOmega plug-in, and then create a new profile. Select socks5 and port 1080 for the proxy server, save and enable it. At this time, you should be able to access the website normally. The proxy server will display the domain name and port of the browser version.

summary

  1. First, let's learn: introduction to Go language learning background
  2. Then learn: Go language basic language detailed explanation
    • development environment
    • Basic grammar
    • Standard library
    • Go language practice
  3. Do project
    • Item 1: guessing game
    • Item 2: command line dictionary
    • Item 3: SOCKS5 agent

Next Preview: getting started with day02 go - Engineering Practice

The source code has been uploaded here:

Source code: https://github.com/nateshao/gogogo/tree/master/day01-05-07

Reference link:

  1. [Go language principle and practice learning materials] the third byte beating youth training camp - back end special session: https://juejin.cn/post/7093721879462019102
  2. Go language tutorial: https://www.runoob.com/go/go-tutorial.html

Posted by Nixon on Tue, 31 May 2022 08:27:34 +0530