Writing to an SD card very slow?

[Bambo]

Hi, i'm trying to write samples from an ADC to an SD card at 16,000HZ, but i am getting terrible SD card write speeds (around 8KB/s)?

SD card: https://www.amazon.co.uk/Kingston-SDCS- ... B079GTYCW4
MCU: STM32 L452RE

This code collects and writes samples to the SD card for 1 second, i expect 16,000 samples to be wrote, but after 1 second, only 4608 samples are writte. This means the file.write() call is blocked for a large amount of time, does anyone know how to solve this?

Code: Select all

#include <SPI.h>
#include <SdFat.h>
#include <HardwareSerial.h>
#include <HardwareTimer.h>

HardwareTimer TIMER(TIM2);
HardwareSerial SerialDebug((uint32_t)PA3, (uint32_t)PA2);
SdFat sd;
SdBaseFile file;
uint32_t TIME_STARTED = 0U;

#define SD1 PD2
#define CS2 PC8
#define MOSI1 PA7
#define MISO1 PA6
#define SCK1 PA5

uint32_t SAMPLES_WROTE = 0U;

uint8_t buff[512];
uint16_t buffIndex = 0U;

void samplerTimer_Callback(HardwareTimer * pInTimer)
{
	uint8_t sample = random(0, 255);
	
	buff[buffIndex] = sample;
	
	SAMPLES_WROTE++;	

	buffIndex++;
	if (buffIndex >= 512)
	{	
		file.write(buff, 512);
		buffIndex = 0;
	}
	
	if ((millis() - TIME_STARTED) > 1000U)
	{
		SerialDebug.println(SAMPLES_WROTE);
		TIMER.pause();

		file.close();
	}
}

void setup() 
{
	SerialDebug.begin(9600);
	SerialDebug.println("Starting...");

	pinMode(SD1, OUTPUT);
	digitalWrite(SD1, HIGH);
	
	pinMode(CS2, OUTPUT);
	
    SPI.setSCLK(SCK1); // CS2
    SPI.setMISO(MISO1); // MISO1
    SPI.setMOSI(MOSI1); // MOSI1

    bool sdOK  = sd.begin(CS2, SD_SCK_MHZ(40));
    if (!sdOK)
    {
    	SerialDebug.println("SD CARD FAIL");
    	while(true) { }
    }
    bool fileOK = file.open("T.BIN", O_CREAT | O_WRITE);
    if (!fileOK)
    {
    	SerialDebug.println("FILE FAIL");
    	while(true) { }
    }
        
    TIMER.setMode(1, TIMER_OUTPUT_COMPARE, NC); // remove this for new version of STM32Dino, required version 1.8.0.
    TIMER.setOverflow(16000, HERTZ_FORMAT);
    TIMER.attachInterrupt(samplerTimer_Callback);
    TIMER.resume();
    
    TIME_STARTED = millis();
}

void loop() 
{

}

Update:
Running this program multiple times with 1 SD card results in different writing speeds, see the following:

11:44:04.720 -> Starting...
11:44:05.823 -> 4096 samples
11:47:45.430 -> Starting...
11:47:46.524 -> 13249 samples
11:47:58.151 -> Starting...
11:47:59.243 -> 4608 samples
11:48:17.136 -> Starting...
11:48:18.230 -> 13282 samples
11:48:27.568 -> Starting...
11:48:28.668 -> 4722 samples

I alternates between 13282 samples and 4722 samples over restarts? Weirdly enough, 4096 is the default allocation size.

[fpiSTM]

Doing the SD write and an Serial print in the callback is really not a good option

[Bambo]

Doing the SD write and an Serial print in the callback is really not a good option

Howcome? isn't the callback just a normal method?

[fpiSTM]

Doing the SD write and an Serial print in the callback is really not a good option

Howcome? isn't the callback just a normal method?

No you are in interrupt context.

[ABOSTM]

Hi @Bambo ,
With fpiSTM, we spent some time on your issue and here is our conclusion:
We tested both Arduino SD library and SdFat.h.
Our results are quite stable around 13000 with core V1.8.0 (whatever library)
And around 14000 when we take some recent improvement https://github.com/stm32duino/Arduino_C ... 2/pull/912
but not yet officially released.
1.8.0
16:29:27.503 -> Starting...
16:29:28.563 -> 13201
16:29:31.811 -> Starting...
16:29:32.905 -> 13201
16:29:35.987 -> Starting...
16:29:37.047 -> 13312
16:29:38.107 -> Starting...
16:29:39.167 -> 13201
16:29:45.960 -> Starting...
16:29:47.053 -> 13023
16:29:52.883 -> Starting...
16:29:53.977 -> 13214

1.8.0 + #912
16:31:47.858 -> Starting...
16:31:48.919 -> 14127
16:31:52.366 -> Starting...
16:31:53.426 -> 14131
16:31:55.183 -> Starting...
16:31:56.210 -> 14046
16:31:58.100 -> Starting...
16:31:59.193 -> 14125
16:32:00.221 -> Starting...
16:32:01.347 -> 14127

Also to get stable result we delete file each time before starting: SD write operation is slower when file size is big.

That said, we noticed that, in our case, writing 512 bytes on SD takes about 3 or 4 milliseconds (of course it may depends on the SD itself, ...).
Because the write operation is a blocking API, and because write operation is made in callback (which means in interrupt context), the timer callback will no more be called during those 4 milliseconds. But callback should be called at 16000Hz so about every 62,5microseconds. So we loose 4 milliseconds/62 microseconds = about 64 interruptions at each transfer of 512 bytes. In 1 seconds there are about 30 time 512bytes to transfer. So a total of about 30*64 = 1940 interruptions lost. This explains why we didn't get 16000 bytes written.

It could be possible to be closer to 16000 in condition to move the write operation in the loop (outside of interrupt context). This allow to continue to fill in the buffer while SD write operation occurs. But you need to be very careful, because then write operation which uses SPI bus, will be interrupted by timer callback. And if this interruption is too long (relative to SPI speed) then SPI will not work any more.
We noticed that the random operation used, around 10 microseconds, is enough to break SPI.

Unfortunately we don't understand why you get unstable results : 1 time about 4000, 1 time about 13000

Below our code with Arduino library

Code: Select all

#include <SPI.h>
#include <SD.h>

File myFile;

HardwareTimer TIMER(TIM2);

volatile uint32_t TIME_STARTED = 0U;
volatile uint32_t SAMPLES_WROTE = 0U;

#define BUFF_SIZE 512

uint8_t buff[BUFF_SIZE];
volatile uint16_t buffIndex = 0U;

void samplerTimer_Callback(HardwareTimer *)
{
  buff[buffIndex++] = random(0,255);
  SAMPLES_WROTE++;
  if (buffIndex >= BUFF_SIZE)
  {
    myFile.write(buff, BUFF_SIZE);
    buffIndex = 0;
  }

  if ((millis() - TIME_STARTED) > 1000U)
  {
    TIMER.pause();
    Serial.println(SAMPLES_WROTE);
    myFile.close();
    while(1);
  }
}

void setup()
{
  Serial.begin(9600);
  Serial.println("Starting...");
  pinMode(LED_BUILTIN, OUTPUT);
  if (!SD.begin(10))
  {
    Serial.println("SD CARD FAIL");
    while (true) { }
  }
  
  if (SD.exists("test.txt")) {
    SD.remove("test.txt");
  }

  myFile = SD.open("test.txt", FILE_WRITE);

  if (!myFile)
  {
    Serial.println("FILE FAIL");
    while (true) { }
  }

  TIMER.setMode(1, TIMER_OUTPUT_COMPARE, NC); // remove this for new version of STM32Dino, required version 1.8.0.
  TIMER.setOverflow(16000, HERTZ_FORMAT);
  TIMER.attachInterrupt(samplerTimer_Callback);
  TIME_STARTED = millis();
  TIMER.resume();

}

void loop()
{

}