High-frequency and low-frequency components of pictures Introduction and related blogs

Low-frequency and high-frequency components of pictures

The low-frequency and high-frequency signals in the image are also called low-frequency components and high-frequency components.

Low-frequency components correspond to areas where the brightness of objects in the image is uniform or changes slowly;

The high-frequency components correspond to the edges, details, and noise of objects in the image.

definition

The frequency of the image: the index of the violent change of the gray value of the image is the gradient of the gray level in the plane space.

low frequency

The low-frequency component in the image signal refers to the place where the image intensity (brightness, grayscale) changes smoothly. That is, a large color block, where the change is not so obvious.

Low frequency means that the color changes slowly, that is, the grayscale changes slowly, which means that it is a continuous gradient area, and this part is the low frequency. For an image, the low frequency is the one that removes the high frequency, that is, the content within the edge is the low frequency, and the content within the edge is most of the information of the image, that is, the general outline and contour of the image, which is the approximate information of the image.

high frequency

The high-frequency component in the image signal refers to the place where the intensity (brightness/grayscale) of the image signal changes drastically, which is what we often call the edge (contour).

High frequency means that the frequency changes rapidly. When does grayscale change rapidly in an image? That is, the grayscale difference between adjacent areas is very large, which means that the change is fast. In an image, there is usually a significant difference between the edge of an image and the background, that is to say, where the edge changes, the grayscale changes rapidly, that is, the part with high frequency of change. Therefore, the gray value of the edge of the image changes quickly, which corresponds to the high frequency, that is, the edge of the image is displayed at high frequency. The details of the image also belong to the area where the gray value changes sharply. It is precisely because of the sharp change of the gray value that the details appear.

In addition, the same is true for noise (that is, noise). At the position of a pixel, the reason for noise is that it is different from the normal point color, which means that the gray value of the pixel is obviously different, that is, gray The degree has changed rapidly, so it is the high frequency part, so there is noise in the high frequency.

example

1. If there is a line of words on a piece of white paper, then we must focus directly on the text without paying too much attention to the white paper itself, where the text is a high-frequency signal, and the white paper is a low-frequency signal.

2. Why is the frequency of an image an indicator to characterize the intensity of grayscale changes in an image? A large area of desert is an area with slow grayscale changes in the image, and the corresponding frequency value is very low, which belongs to low frequency; while for the edge area where the surface attribute changes violently, it is an area with a sharp grayscale change in the image, and the corresponding frequency Higher values are high frequencies.

3. The low frequency retains the light, shadow and color, and the high frequency retains the texture and texture.

4. The man in the striped shirt

In the image above, the scarf and striped shirt are high frequency regions of the image, which change very rapidly from one brightness to another. Higher up in the same image, we see parts of the sky and background that change very slowly, the low-frequency parts of the image.

Spatial and Frequency Domains & Fourier Transforms

Spatial and Frequency Domains

The spatial domain refers to the image plane itself, and methods such as spatial filtering directly process the pixels of the image. Frequency domain filtering is a type of transform domain filtering. It refers to transforming the image (frequency domain refers to after Fourier transform), and processing (filtering) the transform coefficients of the image in the transform domain. Then perform inverse transformation to obtain the filtered image.

How to switch image from spatial domain to frequency domain? Use the Fourier transform.

The spatial domain of the image is the two-dimensional plane coordinate system, which has two orthogonal axes, namely the x and y axes. The magnitude of a point in the spatial domain is the grayscale at that point. The grayscale of a point on the image is the superposition of the grayscale of the point in the x and y directions. In this way, taking x and y as the abscissa and the gray value as the ordinate, respectively, two functions are obtained. Indicates the change of the gray value of the image in the x and y directions.

Fourier series. With knowledge of the Fourier series, the function can be written as the sum of multiple sinusoidal functions of different amplitudes and frequencies. Therefore, through the Fourier transform, the grayscale-x function is transformed into an amplitude-u function, and the grayscale-y function is transformed into an amplitude-v function. Establish a plane coordinate system with u, v as the orthogonal axes, so that the frequency map is obtained.

Since a two-dimensional DFT is usually a complex function, it can be expressed in polar form:

Purpose of Fourier Transform

Question1: What is the purpose of Fourier transforming the image to obtain the spectrogram?

Answer1:

Fourier transform can obtain high frequency and low frequency, and can achieve different purposes for high frequency and low frequency processing;
The Fourier transform is reversible, and the original image can be restored through the inverse Fourier transform;
If the high and low frequencies are processed and then the inverse Fourier transform is performed, the processing will be reflected on the inverse transformed image.

Question2: How to remove high frequency or low frequency information?

Answer2: How to understand frequency domain processing of images? - Zhihu (zhihu.com)

processed by a low pass filter. Keep low frequency information and remove high frequency information. Part of the image detail is lost and the image becomes blurred.

After high-pass filter processing, that is, the low-frequency information in the frequency domain is removed, and only the high-frequency information is saved. Only the details are preserved in the image.

The left image is the frequency graph, the right image is the original image

After low-pass filter processing, high-frequency information is removed and low-frequency information is retained;
The left image is the frequency map after processing, and the right image is the processed image.

After high-pass filter processing, low-frequency information is removed and high-frequency information is retained;
The left image is the frequency map after processing, and the right image is the processed image.

Fourier transform

Image Fourier transform principle python implementation - I firmly believe that the sun is shining - Blog Park

recommend Digital Image Processing - Tags - I Believe The Sun Is Bright - Blog Garden (cnblogs.com)

In digital image processing, two classic transforms are widely used - Fourier transform and Hough transform. Fourier transform is to convert the signal in the time domain into a signal in the frequency domain, and then perform image denoising, image enhancement and other processing.

After Fourier Transform (FT), the viewing angle of the same thing changes accordingly, and some features that are not easily discernible from the time domain can be found in the frequency domain. Some things that are hard to deal with in the time domain can be easily dealt with in the frequency domain.

Fourier's Theorem: "Any continuous periodic signal can be represented (or infinitely approximated) as a superposition of a series of sinusoidal signals."

python method to implement Fourier transform

See Fourier Transform of Image Processing - Zhihu (zhihu.com)

There are two common methods in python: numpy and OpenCV.

1.numpy

Python Numpy np.fft2() method | Geek Tutorial (geek-docs.com)

NumPy 1.22 Chinese (runebook.dev)

numpy.fft.fft(a, n=None, axis=-1, norm=None)

numpy.fft.fft - compute the 1D discrete Fourier transform. (runebook.dev)

Function: Calculate the one-dimensional Fourier transform. Computes a one-dimensional n-point discrete Fourier transform (DFT) using an efficient Fast Fourier Transform (FFT) algorithm [CT].

numpy.fft.fft2(a, n=None, axis=-1, norm=None)

numpy.fft.fft2 - compute the 2D discrete Fourier transform. (runebook.dev)

Function: Calculate the two-dimensional Fourier transform. This function computes the n-dimensional discrete Fourier transform on any axis in an M-dimensional array by means of a Fast Fourier Transform (FFT). By default, the transformation is done on the last two axes of the input array, i.e. a 2D FFT.

numpy.fft.fftn()

numpy.fft.fftn - compute the N-dimensional discrete Fourier transform. (runebook.dev)

Function: Calculate the Fourier transform of n dimensions. Computes the N-dimensional discrete Fourier transform on any number of axes in an M-dimensional array via the Fast Fourier Transform (FFT).

numpy.fft.fftfreq()

numpy.fft.rfftfreq - returns the sampling frequency of the discrete Fourier transform (used with rfft, irfft). (runebook.dev)

Function: Returns the sampling frequency of the discrete Fourier transform (used with rfft, irfft). The returned floating-point array f contains the frequency bin centers in cycles per unit sample interval (zero at the beginning). For example, if the sample interval is seconds, the frequency unit is cycles/second.

numpy.fft.shift()

numpy.fft.fftshift - shift the zero frequency component to the center of the spectrum. (runebook.dev)

Function: Move the DC component in the FFT output to the center of the spectrum. All axes listed swap half-width spaces (default is all).

20*np.log(np.abs(fshift))

Function: The result obtained by Fourier transform is an array of complex numbers, which cannot be directly used to display images. To get a spectral grayscale image, we need a mapping that maps complex numbers between [0, 255].

np.abs(i_img)

Function: Set the value range and realize the inverse Fourier transform

2.OpenCV

cv2.dft()

Function: implement Fourier transform

cv2.magnitude( parameter 1, parameter 2)

The function of this function is the same as that of numpy arrays: np.abs(fshift) is the same thing, np.abs(fshift) is to find the size of each complex number in the complex number array fshift (Note: fshift is a one-dimensional complex number array), and cv2.dft() returns a two-dimensional array (one dimension is the real part of the complex number, and one is the imaginary part of the complex number).

cv2.idft(ishift)

Function: Realize the inverse Fourier transform.

Application example: high and low frequency image mixing

How to understand the high and low frequency of the image?

The low-frequency parts of the image can be understood as "contours", such as the face shape of a human face.

The high-frequency parts of the image can be understood as "details", such as wrinkles, spots, etc. of the human face.

Therefore, after blurring the image, the low-frequency part of the image is obtained, and sharpening the image will make the high-frequency information of the image more.

Zoom in and out of the image to see different results?

When we zoom in on the mixed image, high-frequency information can be clearly seen, so the human eye is more inclined to recognize this part of the information. Conversely, when we zoom out the image, the high frequency information is not so easy to be observed, at this time we are more inclined to identify the low frequency part.

Here's an aliased image of Harry Potter and Voldemort. As you can see, when the image is zoomed in we see Harry Potter, and when the image is zoomed out, we think the image is depicting Voldemort.

How to achieve the effect of blending images?

The general idea of the implementation is simple - overlay a picture with only low frequency information and an image with only high frequency information. Specific steps are as follows:

Take the first image:
1. Prepare low-frequency filters (usually Gaussian blur)
2. Convert the image into a matrix and layer it (for example, a JPG image will be divided into three layers of RGB) for processing. Among them, the first two columns represent the horizontal and vertical coordinates of the image, and the third column stores the index of different layers (for example, JPG has 0-2, PNG has 0-3 (the fourth layer (3) stores transparency)); For example, [;, ;, 0] refers to the data points of the R layer
3. Each dimension of the image is convolved with a Gaussian filter
Take the second image:
1. Prepare low-frequency filters (usually Gaussian blur)
2. convert image to matrix
3. Each dimension of the image is convolved with a Gaussian filter
4. Subtract the matrix after the rolling machine from the original image matrix to get the high-frequency image matrix
Add the two processed image matrices to get the mixed image

Implementation code: GitHub - MinisculeDust/Hybrid-Image

Implementation code

MyHybridImages.py

import numpy as np
from MyConvolution import convolve
# version 3.3 2019.11.7-18:57
# https://github.com/MinisculeDust/Hybrid-Image/blob/master/MyHybridImages.py


def makeGaussianKernel(sigma):

    # set the sacle of templet
    size = (int)(8 * sigma + 1) # (this implies the window is + / - 4 sigmas from the centre of the Gaussian)
    if (size % 2 == 0):
        size += 1   # size must be odd

    # templet's centre position
    center = (size - 1) / 2

    # state kernel
    kernel = np.zeros((size, size))

    kernel_sum = 0
    # gaussian calculating
    for i in range(size):
        x2 = pow(i - center, 2)
        for j in range(size):
            y2 = pow(j - center, 2)
            g = np.exp(-(x2 + y2)/(2 * sigma * sigma)) / (2 * np.pi * sigma * sigma)
            kernel[i][j] = g
            kernel_sum += kernel[i][j]

    # normalisation
    # kernel_sum2 = 0
    # for i in range(size):
    #     for j in range(size):
    #         kernel[i][j] = kernel[i][j] / kernel_sum
    #         kernel_sum2 += kernel[i][j]

    return kernel

def myHybridImages(lowImage: np.ndarray, lowSigma, highImage: np.ndarray, highSigma):

    # make kernel
    low_kernel = makeGaussianKernel(lowSigma)
    high_kernel = makeGaussianKernel(highSigma)

    # convolve low-pass pictures
    low_image = convolve(lowImage, low_kernel)

    # make high-pass picture
    high_image = (highImage - convolve(highImage, high_kernel))

    # final picture
    # the weights between and final lighting can be changed flexibly
    weight = 1
    weight2 = 1
    adjustment = 0
    hybrid_image =  high_image * weight2 + low_image * weight + adjustment
    # hybrid_image = high_image + low_image

    # randomly double check the output
    # print(hybrid_image[11][22][1])
    # print(hybrid_image[44][55][0])
    # print(hybrid_image[357][159][2])

    return hybrid_image

MyConvolution.py

import numpy as np
# version 3.4 2019.11.8-9:37
# https://github.com/MinisculeDust/Hybrid-Image/blob/master/MyConvolution.py

def calculate_convolution(image, kernel):

    # rotating kernel with 180 degrees
    kernel = np.rot90(kernel, 2)

    kernel_heigh = int(np.array(kernel).shape[0])
    kernel_width = int(np.array(kernel).shape[1])

    # set kernel matrix to random int matrix
    if ((kernel_heigh % 2 != 0) & (kernel_width % 2 != 0)):  # make sure that the scale of kernel is odd
        # the scale of result
        conv_heigh = image.shape[0] - kernel.shape[0] + 1
        conv_width = image.shape[1] - kernel.shape[1] + 1
        conv = np.zeros((conv_heigh, conv_width))

        # convolve
        for i in range(int(conv_heigh)):
            for j in range(int(conv_width )):
                result = (image[i:i + kernel_heigh, j:j + kernel_width] * kernel).sum()
                # if(result<0):
                #     resutl = 0
                # elif(result>255):
                #     result = 255
                conv[i][j] = result
    return conv

def convolve(image: np.ndarray, kernel: np.ndarray) -> np.ndarray:

    # zero padding
    kernel_half_row = int((kernel.shape[0]-1)/2)
    kernel_half_col = int((kernel.shape[1]-1)/2)

    # judge how many channels
    if len(image.shape) == 3:
        image = np.pad(image, ((kernel_half_row, kernel_half_row), (kernel_half_col, kernel_half_col),(0, 0)), 'constant', constant_values=0)

        # if image.shape[2] == 3 or image.shape[2] == 4:
        # if style is png, there will be four channels, but we just need to use the first three
        # if the style is bmp or jpg, there will be three channels
        image_r = image[:, :, 0]
        image_g = image[:, :, 1]
        image_b = image[:, :, 2]
        result_r = calculate_convolution(image_r, kernel)
        result_g = calculate_convolution(image_g, kernel)
        result_b = calculate_convolution(image_b, kernel)
        result_picture = np.dstack([result_r, result_g, result_b])
    # if the picture is black and white
    elif len(image.shape) == 2:
        image = np.pad(image, ((kernel_half_row, kernel_half_row), (kernel_half_col, kernel_half_col)), 'constant', constant_values=0)
        result_picture = calculate_convolution(image, kernel)

    # returns the convolved image (of the same shape as the input image)
    return result_picture


def fourier_trans(image: np.ndarray, kernel: np.ndarray):
    # make the scale of the kernel as the same as pictures
    # make sure it can work for different sance of pictures
    if (image.shape[0] - kernel.shape[0]) % 2 == 0:
        pad_heigh = np.int(((image.shape[0] - kernel.shape[0])) / 2)
    else:
        pad_heigh = np.int(((image.shape[0] - kernel.shape[0])) / 2) + 1

    if (image.shape[1] - kernel.shape[1]) % 2 == 0:
        pad_width = np.int(((image.shape[1] - kernel.shape[1])) / 2)
    else:
        pad_width = np.int(((image.shape[1] - kernel.shape[1])) / 2) + 1
    pad_heigh_light = np.int(((image.shape[0] - kernel.shape[0])) / 2)
    pad_width_light = np.int(((image.shape[1] - kernel.shape[1])) / 2)
    kernel = np.pad(kernel, ((pad_heigh_light, pad_heigh), (pad_width_light, pad_width)), 'constant', constant_values=0)
    print("kernel.shape", kernel.shape)
    copy_fft2_image = np.zeros(image.shape)

    # fourier transform for kernel
    # shift the centre of kernel to axis origin and then do fourier transform
    fft2_kenel_after = np.fft.fft2(np.fft.fftshift(kernel))

    if len(image.shape) == 3:
        # fourier transform
        for i in range(image.shape[2]):
            image_fft = np.fft.fft2(image[:, :, i])
            # print("fft2_kenel_after * image_fft.shape ==== ", (fft2_kenel_after * image_fft).shape)
            # image_fft = np.fft.fftshift(np.fft.fft2(image[:, :, i]))
            # copy_fft2_image[:, :, i] = np.fft.ifftshift(np.fft.ifft2(fft2_kenel_after * image_fft))
            frequcy_result = fft2_kenel_after * image_fft
            # copy_fft2_image[:, :, i] = np.fft.fftshift(np.fft.ifft2(frequcy_result))
            copy_fft2_image[:, :, i] = np.fft.ifft2(frequcy_result)
    elif len(image.shape) == 2:
        image_fft = np.fft.fft2(image)
        frequcy_result = fft2_kenel_after * image_fft
        copy_fft2_image[:, :] = np.fft.ifft2(frequcy_result)

    return copy_fft2_image