OV7670, Generic STM32F103RC and ILI9341 display

What are you developing?
User avatar
RogerClark
Posts: 6877
Joined: Mon Apr 27, 2015 10:36 am
Location: Melbourne, Australia
Contact:

Re: OV7670, Generic STM32F103RC and ILI9341 display

Post by RogerClark » Fri Sep 22, 2017 8:09 am

Just did some speed tests and 1000 calls to readPixels

Code: Select all

  unsigned long m=millis();
  for(int i=0;i<1000;i++)
  {
    tft.readPixels(0,0,320,0, lineBufLCD);
  }
  Serial.println(millis()-m);
shows 1284ms

So about 1.2ms per line.

This was slower than I'd expected and looking in the code for readPixels, I think this is caused because the red, green and blue of each pixel seem to be read by separate calls to spiread()

I presumed the whole line was being read as 16 bit pixels, but it looks like it reads as pseudo 24 bit and those values are packed into RGB 565 by Steve's function

I need the values in 24 bit, so I will probably write a new function e.g. readPixels24bit , which will be faster as I presume the whole window buffer can be read from the display in one SPI read

stevestrong
Posts: 1601
Joined: Mon Oct 19, 2015 12:06 am
Location: Munich, Germany

Re: OV7670, Generic STM32F103RC and ILI9341 display

Post by stevestrong » Fri Sep 22, 2017 12:29 pm

Just for the record, here an example how to write BMP: http://forum.arduino.cc/index.php?topic ... #msg849962

and a nice JPEG encoder (for STM32L1): https://github.com/mitchd/vt-fox-1/tree ... de/encoder

User avatar
RogerClark
Posts: 6877
Joined: Mon Apr 27, 2015 10:36 am
Location: Melbourne, Australia
Contact:

Re: OV7670, Generic STM32F103RC and ILI9341 display

Post by RogerClark » Fri Sep 22, 2017 8:19 pm

Thanks Steve

I am already using snippetsnof the BMP code from the Arduno forum, and it seems to work OK.

Thanks for the link to the L1 joeg compressor, but for the F1 I am not sure if it has enough RAM.

Writing to SD is currently very slow if I only write 1 line at a time. ( over 5 seconds)

I should be able to write 12 lines at a time, or potentially even 15 lines, but I have not made that work yet.

My initial calculations were that I could write the 320x240 display to SD in approx, 400mS , but I had not factored the rather slow transfer speed to the SD, whuch is only a few hundred kIlo bytes per second.
So I think the time to write to SD is likely to be around 900mS or perhaps 800mS at best.

User avatar
RogerClark
Posts: 6877
Joined: Mon Apr 27, 2015 10:36 am
Location: Melbourne, Australia
Contact:

Re: OV7670, Generic STM32F103RC and ILI9341 display

Post by RogerClark » Fri Sep 22, 2017 10:07 pm

By writing 12 lines at a time, I've got the time down to 742 mS

I'm still reading the display, line by line, so I can probably shave a few mS off this, but although reading from the LCD can be faster, because the image seems to be upside down, I may still need to flip it vertically in code (in blocks of 12 lines at a time)

BTW.
This harks back to my first job, when I worked on high resolution image processing for the print industry, in the days before a 50mB image could easily be processed by a $10 RPi Zero ;-)

Edit

I did some timing tests, and reading multiple lines from the LCD doesnt look like it will save much time, and is probably not worth the bother at the moment

Time to read 240 lines individually from the LCD would be 116.64 mS
Time to read 240 lines in blocks of 10 from the LCD would be 104.13 mS

However, if I read 10 lines from the display the line(s) buffer would need to be processed to swap the line order, and this is going to add a small amount of time to the total.

It would probably be marginally quicker to do this but it would be less than 10mS on a total time of around 700mS, so at the moment, I don't think its worth the time to get it working !

PS. I also tried change the Window params to that the window Y direction of the window was -1 e.g. lines 239 to 230 rather than 230 to 239
However, this seemed to result in corrupt image data
(Note I'd need to retest this as I'm also having SPI issues when I attempt to transfer more than 11 lines at a time, so I can't rule out that being the problem rather than an inverted image window problem)

User avatar
RogerClark
Posts: 6877
Joined: Mon Apr 27, 2015 10:36 am
Location: Melbourne, Australia
Contact:

Re: OV7670, Generic STM32F103RC and ILI9341 display

Post by RogerClark » Sat Sep 23, 2017 12:13 am

I just double checked the if the readback can have yStart > yEnd but it definitely causes corruption of the data

I'm going to go back to reading the LCD line by line, as I don't think its going to save any significant time by reading multiple lines because of the time to swap lines, i.e there would need be 3 memcpy calls per pair of lines that are swapped and it would need an additional line of buffering which could not be used for the SD card transfer

victor_pv
Posts: 1642
Joined: Mon Apr 27, 2015 12:12 pm

Re: OV7670, Generic STM32F103RC and ILI9341 display

Post by victor_pv » Sat Sep 23, 2017 2:56 am

Roger how large are the buffers you read from the camera, and how large the one you write to the sdcard?

I ask because I have had good luck running the sdfat in parallel with other things with a modified driver that uses the new DMA callback functions, and FreeRTOS.
Obviously if the buffers are small, there is no much gain, since the RTOS has to switch tasks and that wastes some cpu cycles, but if the buffers are large enough, say some KB, you may be able to do more things in parallel, perhaps read lines from the camera and write to the card only taking the time of the longest one of them, while the other is ran in the background using cpu time that would be wasted.
It's pretty similar to what you were doing with the async function with the leds, but since there are callbacks, there is no need to check if the DMAs are over, the spi driver will just call the corresponding ISR when they are.

It my case is just a proof of concept with the WAV player, and goes like this:
Task 1 reads from the SDcard, and sets the data as buffer for another another DMA channel to output to timers.
Task 2 writes some stats to the screen (cpu usage mostly).
Task 3 just draws a cube just to do something with the cpu time.
Some interrupt routines switching buffers.

So when task 1 was before "blocking" in sdfat reading, it doesn't block anymore and instead does a task switch to task 2 or 3, whichever is ready to execute.
When the spi DMA completes calls a new ISR that tells the RTOS the sdfat task, task1, is ready to execute, and the rtos does a switch back to that task.
Since that's managed within the sdfat spi driver code, it's compatible with anything, the other tasks do not need to have any change, they just get executed more frequently.
I forgot how much cpu time I could see saved that way, but it was a few % points, and my code is just reading from the car, writes take even longer so may benefit more.
You could use the same methods for both writing to the card, and reading from the camera, so while both transfers are going in parallel, whatever code needs to run in the cpu can be running.

User avatar
RogerClark
Posts: 6877
Joined: Mon Apr 27, 2015 10:36 am
Location: Melbourne, Australia
Contact:

Re: OV7670, Generic STM32F103RC and ILI9341 display

Post by RogerClark » Sat Sep 23, 2017 3:52 am

Hi Victor

Both the camera and the SD are on the same SPI bus (SPI1)
I can't use SPI2 because the camera has to be on specific pins i.e PB8 - PB15 as Steve's code does a DMA read from the upper half of port B.
(and thats where SPI 2 is located)

I'll need to check if SPI2 can be remapped, but I've got very few pins unused, so it may not be possible to remap as there are some other pins in the circuit which cant be changed because the timers that they are associate with.

I'm not to concerned if it takes 750mS to save to SD, its just strange that I get a sudden slowdown if I use a 12 line buffer, when a 11 buffer is OK.

User avatar
RogerClark
Posts: 6877
Joined: Mon Apr 27, 2015 10:36 am
Location: Melbourne, Australia
Contact:

Re: OV7670, Generic STM32F103RC and ILI9341 display

Post by RogerClark » Sat Sep 23, 2017 4:54 am

Steve

I've tried to save the LCD display after a frame has been read from the camera, but it fails because it looks like the timer or DMA etc is interfering with the SD SPI access somehow.

Here is my code that I'm using to read the display and write to SD

Code: Select all

#include <SdFat.h>
#include "Adafruit_ILI9341_STM.h"

#define TFT_CS         PB0                  
#define TFT_DC         PA2               
#define TFT_RST        PB1 
Adafruit_ILI9341_STM tft = Adafruit_ILI9341_STM(TFT_CS, TFT_DC, TFT_RST); // Use hardware SPI

const uint8_t SD_CS = PC15;
SdFat sd;

unsigned long testText()
{
  tft.fillScreen(ILI9341_BLACK);
  tft.setCursor(0, 0);
  tft.setTextColor(ILI9341_RED);  tft.setTextSize(3);
  tft.println("LCD Screen reader\n");
  tft.setTextColor(ILI9341_GREEN); tft.setTextSize(3);
  tft.println("rogerclark.net\n");
  tft.setTextColor(ILI9341_BLUE); tft.setTextSize(3);
  tft.println(__TIME__);
  tft.setTextSize(2);

  return 0;
}

void LCD2SD()
{
const int w = 320;
const int h = 240;
SdFile file;  
#define NUM_LINES_BUFFERED 12
uint8_t lineBufSD[w*3*NUM_LINES_BUFFERED];

  unsigned char bmpFileHeader[14] = {'B', 'M', 0, 0, 0, 0, 0, 0, 0, 0, 54, 0, 0, 0 };
  unsigned char bmpInfoHeader[40] = {40, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 24, 0 };  
  unsigned long m;
  char name[] = "LCD_00.bmp";


  // if name exists, create new filename
  for (int i = 0; i < 100; i++)
  {
    name[4] = i / 10 + '0';
    name[5] = i % 10 + '0';
    if (file.open(name, O_CREAT | O_EXCL | O_WRITE))
    {
      break;
    }
  }

  // create image data
  int filesize = 54 + 4 * w * h;      //  w is image width, h is image height
  bmpFileHeader[ 2] = (unsigned char)(filesize    );
  bmpFileHeader[ 3] = (unsigned char)(filesize >> 8);
  bmpFileHeader[ 4] = (unsigned char)(filesize >> 16);
  bmpFileHeader[ 5] = (unsigned char)(filesize >> 24);

  bmpInfoHeader[ 4] = (unsigned char)(       w    );
  bmpInfoHeader[ 5] = (unsigned char)(       w >> 8);
  bmpInfoHeader[ 6] = (unsigned char)(       w >> 16);
  bmpInfoHeader[ 7] = (unsigned char)(       w >> 24);
  bmpInfoHeader[ 8] = (unsigned char)(       h    );
  bmpInfoHeader[ 9] = (unsigned char)(       h >> 8);
  bmpInfoHeader[10] = (unsigned char)(       h >> 16);
  bmpInfoHeader[11] = (unsigned char)(       h >> 24);

  file.write(bmpFileHeader, sizeof(bmpFileHeader));    // write file header
  file.write(bmpInfoHeader, sizeof(bmpInfoHeader));    // " info header

  m=millis();
  uint8_t t;
  int lineOffset;
  for (int y = 0 ; y < h ; y++) 
  {

    lineOffset = y%NUM_LINES_BUFFERED *3 * w;
    tft.readPixels24(0,(h-1)-y,320,(h-1)-y, lineBufSD + lineOffset);
    if ((y+1)%NUM_LINES_BUFFERED==0)
    {
      // swap colour channels from  RGB to BGR
      for (int x = 0; x < w * 3 * NUM_LINES_BUFFERED; x+=3) 
      {
        t=lineBufSD[x+2];
        lineBufSD[x+2]=lineBufSD[x];
        lineBufSD[x]=t;
      }
       file.write(lineBufSD, 3 * w * NUM_LINES_BUFFERED);
    }
  }
  Serial.print("Saved in ");Serial.print(millis()-m);Serial.println(" mS ");
  file.close();  
}

void setup() 
{
  Serial.begin(9600);
  delay(500);
  Serial.println("Starting");

  if (!sd.begin(SD_CS, SD_SCK_MHZ(50)))
  {
    sd.initErrorHalt();
  }
  
  tft.begin();
  uint16_t screen_w = ILI9341_TFTHEIGHT;
  uint16_t screen_h = ILI9341_TFTWIDTH;

  tft.setRotation(1);
  testText();// Draw something
 // tft.setAddrWindow(0, 0, screen_w, screen_h);  
//  tft.setCursor(0, 0);
  LCD2SD();
}

void loop()
{  
}
Also.

I've noticed that if I call

tft.setAddrWindow(0, 0, screen_w, screen_h);

It interferes with reading of the LCD even though the readPixels sets the ILI9341_CASET and ILI9341_PASET registers

What happens is the first few lines of the display get corrupted.

This effect with the the display getting corrupted, only seems to happen if I call setAddrWindow just prior to readPixels24
Some other actions in between e.g. writing some text to the LCD, it seems OK.

I notice that you do at "read GRAM" and then "dummy read" in the comments, so I wonder whether something else has to happen after setting the address window, because it sends the command

writecommand(ILI9341_RAMWR); // write to RAM

User avatar
RogerClark
Posts: 6877
Joined: Mon Apr 27, 2015 10:36 am
Location: Melbourne, Australia
Contact:

Re: OV7670, Generic STM32F103RC and ILI9341 display

Post by RogerClark » Sat Sep 23, 2017 5:20 am

Another update.

It looks like the Timer setup code in the LiveOV7670 code

Code: Select all

void TIMER_Setup(void)
{
	gpio_set_mode(GPIOA, 1, GPIO_INPUT_FLOATING);

// Slave mode: Reset mode (see RM0008, chap. 15.3.14, page 394)
// ------------------------
// The counter and its prescaler can be reinitialized in response to an event on a trigger input.
// Moreover, if the URS bit from the TIMx_CR1 register is low, an update event UEV is generated.
// Then all the preloaded registers (TIMx_ARR, TIMx_CCRx) are updated.
//
// In the following example, the upcounter is cleared in response to a rising edge on TI2 input:
// • Configure the channel 2 to detect rising edges on TI2.
// - Configure the input filter duration (in this example, we don’t need any filter, so we keep IC1F=0000).
// - The capture prescaler is not used for triggering, so you don’t need to configure it.
// - The CC2S bits select the input capture source only, CC2S = 01 in the TIMx_CCMR1 register.
// - Write CC2P=0 in TIMx_CCER register to validate the polarity (and detect rising edges only).
// • Configure the timer in reset mode by writing SMS=100 in TIMx_SMCR register.
// - Select TI2 as the input source by writing TS=101 in TIMx_SMCR register.
// • Start the counter by writing CEN=1 in the TIMx_CR1 register.
//
// The counter starts counting on the internal clock, then behaves normally until TI2 rising edge.
// When TI2 rises, the counter is cleared and restarts from 0.
// In the meantime, the trigger flag is set (TIF bit in the TIMx_SR register) and an interrupt request,
// or a DMA request can be sent if enabled (depending on the TIE and TDE bits in TIMx_DIER register).
//
// This event will trigger the DMA to save the content of GPIOB.IDR[1] to memory.

	timer_pause(TIMER2); // stop timer
	timer_init(TIMER2);  // turn timer RCC on
	// configure PA2 = timer 2 channel 2 == input TI2

#define TIMER_RELOAD_VALUE 2 // must be adapted according to the results
	// as this mode is not supported by the core lib, we have to set up the registers manually.
	//(TIMER2->regs).gen->CR1 = TIMER_CR1_CEN;
	(TIMER2->regs).gen->CR2 = 0;
	(TIMER2->regs).gen->SMCR = (TIMER_SMCR_TS_TI2FP2 | TIMER_SMCR_SMS_RESET);//TIMER_SMCR_SMS_TRIGGER);
	(TIMER2->regs).gen->DIER = (TIMER_DIER_UDE); // enable DMA request on TIM2 update
	(TIMER2->regs).gen->SR = 0;
	(TIMER2->regs).gen->EGR = 0;
	(TIMER2->regs).gen->CCMR1 = TIMER_CCMR1_CC2S_INPUT_TI2; // IC2F='0000', IC2PSC='0', CC2S='01' 
	(TIMER2->regs).gen->CCMR2 = 0;
	(TIMER2->regs).gen->CCER = (TIMER_CCER_CC2P); // inverse polarity, active low
	(TIMER2->regs).gen->CNT = 0;//TIMER_RELOAD_VALUE;	// set it only in down-counting more
	(TIMER2->regs).gen->PSC = 0;
	(TIMER2->regs).gen->ARR = 0;//TIMER_RELOAD_VALUE;
	(TIMER2->regs).gen->CCR1 = 0;
	(TIMER2->regs).gen->CCR2 = 0;
	(TIMER2->regs).gen->CCR3 = 0;
	(TIMER2->regs).gen->CCR4 = 0;
	(TIMER2->regs).gen->DCR = 0;			// don't need DMA for timer
	(TIMER2->regs).gen->DMAR = 0;
	// don't forget to set the DMA trigger source to TIM2-UP
	//timer_resume(TIMER2); // start timer
}
is interfering with the transfer to the SD.

The file system gets corrupted.

The DMA has also been set for the camera

Code: Select all

void DMA_Setup(void)
{
	dma_init(DMA1);
	uint32_t pmem = ((uint32_t)(&(GPIOB->regs->IDR)) + 1); // use GPIOB high byte as source
    dma_setup_transfer(DMA1, DMA_CH2,
	    (uint8_t *)pmem, DMA_SIZE_8BITS,
        (uint8_t *)&pixBuf, DMA_SIZE_8BITS,
		(DMA_MINC_MODE));//| DMA_CIRC_MODE));
    dma_set_priority(DMA1, DMA_CH2, DMA_PRIORITY_VERY_HIGH);
    //dma_set_num_transfers(DMA1, DMA_CH2, 2*PIXELS_PER_LINE); // 2 readings for each pixel
    //dma_enable(DMA1, DMA_CH2);
}
But SD still seems to work, after calling DMA_Setup, but not after then calling Timer_Setup

I tried calling timer_pause(TIMER2) prior to calling my function to save to disk, but it didn't make any difference

I've looked to see if SDFat uses any timers, but it doesnt appear to..

I've also tried just calling Timer_Setup() and not calling DMA_Setup and its definitely the timer thats breaking the SD.

I'll try to work out specifically in Timer_Setup() causes this problem

User avatar
RogerClark
Posts: 6877
Joined: Mon Apr 27, 2015 10:36 am
Location: Melbourne, Australia
Contact:

Re: OV7670, Generic STM32F103RC and ILI9341 display

Post by RogerClark » Sat Sep 23, 2017 6:56 am

I'm getting closer to a solution.

The problem is

(TIMER2->regs).gen->DIER = (TIMER_DIER_UDE); // enable DMA request on TIM2 update

Seems to be somehow interfering with the SDFat

If I disable the DMA on TIM2 update, I am able to save a image to SD.

Code: Select all

(TIMER2->regs).gen->DIER = (0); // disable DMA request on TIM2 update
Unfortunately when I re-enabled the DMA , the image from the camera is screwed up.

Pausing and resuming the timer doesnt seem to help...

I think I"m going to need to re-init the DMA and timer each time an image is taken and transferred to SD

Post Reply