CMOS image acquisition systems generally suffer from image quality problems. Without special processing of images, image quality is difficult to guarantee. In recent years, with the rapid development of SoC technology, in the field of image acquisition and processing, SoC image sensors have emerged, which integrate CMOS sensor and graphics processor functions, and can obtain very satisfactory image quality. The video acquisition system designed in this paper uses SoC imaging chip MT9M111 and USB2.0 interface chip CY7C68013.
system structure
The functional block diagram of this system is shown in Figure 1. When the image sensor starts to work, the collected data is first stored in SRAM1 through the FPGA control logic. After the frame image collection / storage process is completed, SRAM1 enters the write end state. At this time, SRAM is switched, and SRAM2 continues to store the collected data. At the same time, SRAM1 is in a readable state and is controlled by the control logic in the FPGA to transfer the data in SRAM1 to the USB chip and then to the host. This system adopts double SRAM structure and ping-pong mechanism, and the two pieces of memory work alternately, so that the image acquisition and transmission are carried out in parallel. The dual-frame memory structure not only improves the speed of the system, but also, because the algorithms for implementing various image processing in FPGA mostly require relatively large storage space, the two large-capacity SRAMs can act as external caches when implementing the algorithms.
MT9M111
This system uses SoC product MT9M111 which integrates CMOS sensor and graphics processor from Micron. MT9M111 is a low-power, low-cost progressive scan CMOS image sensor; 1.3 megapixel resolution (1280H × 1024V); 1 / 3-inch optical format; full resolution 15fps power consumption is 170mW, VGA resolution 30fps power consumption is 90mW. The MT9M111 uses a low-leakage DRAM process and is equipped with Micron ’s patented DigitalClarity technology, which can provide clear and bright color images even under the worst lighting conditions. MT9M111 has lower dark current, and reduces chroma / luminance interference and transient noise. MT9M111's embedded programmable image stream processor provides functions including color recovery and repair, automatic exposure, white balance, lens shadow correction, increased sharpness, programmable grayscale correction, dark level imbalance correction, flicker avoidance, continuous adjustment filter Light size, smooth digital zoom, fast auto exposure mode, and defect correction when not working. Moreover, it is also equipped with a two-wire serial interface, and the USB chip can be configured through the two-wire serial port.
IS61WV20488
The SRAM parameters related to image processing are mainly the read and write speed and capacity of SRAM. In terms of capacity, the maximum resolution of the image collected by this system is 1280 × 1024, the data width is 8 bits, and the 2M × 8bit SRAM can meet the requirements for storing one frame of image data. The read and write speed of SRAM is generally 12ns, 15ns, 20ns or slower. Because the read and write speed of SRAM directly affects the clock of the entire image processing system, the faster the read and write speed of SRAM, the better. This system selects IS61WV20488 of ISSI company, the chip capacity is 2M × 8bit.
CY7C68013
The image data transmission part adopts the interface chip CY7C68013 specially used by USBpress for the USB2.0. The chip includes an enhanced 8051 processor with 815kB on-chip RAM (compatible with the standard 8051 series, speed increased by 3 "5 times), 4kB FIFO memory and general programmable interface I2C bus, serial interface engine (S IE) and USB2. 0 Transceiver.
Figure 1 Video acquisition system based on SoC image sensor
System software / hardware design
The system software / hardware design consists of three parts: image acquisition / storage module, image transmission module, and USB driver / host application module.
Image acquisition / storage module
This module mainly transfers the image data collected by the imaging chip MT9M111 to SRAM in real time by the control logic of FPGA. In the system, a dual-frame storage structure is adopted, each of which is composed of a piece of IS61WV20488 SRAM, and can store one frame of image data with 1280 × 1024 resolution. Due to the ping-pong mechanism, the two memories work alternately, so that the image acquisition and transmission are performed in parallel. To ensure that only one SRAM can read the collected image data at any time, a read mutex is set. Similarly, only one SRAM can receive the collected image data. Therefore, a write mutex is also set. It should be noted that, because the image sensor's image data output speed is slower than the USB2.0 transfer speed, after reading the data of SRAM2, you need to wait for another SRAM1 to complete writing image data before writing to SRAM2 One frame of image data, and SRAM1 can directly perform the work of reading image data without waiting. This cycle back and forth realizes parallel work and effectively improves the work efficiency of the system. Figure 2 is the functional block diagram of the system control circuit, each module inside the FPGA is written in Verilog HDL.
Figure 2 FPGA control circuit block diagram
Image transmission module
After collecting a frame of images, the image data in the external RAM is read into the host through the USB clock signal control module. In this image acquisition system, the slave FIFO of CY7C68013 is used to work asynchronously, and the FIFO is configured to be connected to the EP2 port, each data packet is 1024 bytes, 4 buffers, block transmission mode. This setting can meet the system requirements, and also effectively use the internal 4kB FIFO to transmit the collected image data. The system control uses the FALGB signal pin to report the status of "FIFO full". The default is low level effective. This article uses the automatic input method. When the data in the FIFO is full of a certain amount, EZ-USB-FX2 will directly transfer the data to the USB transceiver through the FIFO without CPU intervention, thus increasing the transmission speed. The system starts sending automatically when the FIFO is full 1kB.
USB driver and host application module
The development of the USB device driver is the difficulty of the USB system development, especially in the case of large data transmission and high speed requirements in this system, it is necessary to write an efficient USB device driver to ensure the real-time transmission of high-resolution images . This system uses DDK to develop WDM driver. In fact, the USB client driver contains a large number of routines, which is very helpful to the development of the driver. The main function of the host application is to read image data through the USB interface and display dynamic images in real time. To improve the efficiency of the host application, dual threads can be used.
Conclusion
This system uses the MT9M111 image sensor with 1.3 million pixels to ensure the image quality. The USB2.0 interface chip CY7C68013 is used to ensure the real-time transmission of images, and the design is flexible, providing software / hardware support for implementing various image processing algorithms.
Guangzhou Ehang Electronic Co., Ltd. , https://www.ehangmobile.com