How to calculate Time Delay Estimation?

[Nate Duong]

I have 2 data files, which links are attached below:
Transmitted data
https://www.dropbox.com/s/0nmhw6mpgh7upmv/TX.dat?dl=0
Received data
https://www.dropbox.com/s/xgyo6le3bcmd25r/RX.dat?dl=0

Those binary data are read by this MATLAB code:
%% initial values:
nsamps = inf;
nstart = 0;
%% input files
file_tx = 'test1-qpsk-8-8000.0-ksps.dat';
file_rx_tx_on = 'tx30-rx50-2017-03-29-14.48.56-utc.dat';
%% read files
x_tx = readcplx(file_tx, nsamps,nstart);
x_rx_on = readcplx(file_rx_tx_on, nsamps,nstart);
correlation = xcorr(x_tx,x_rx_on);

%% readcplx function:

function x = readcplx(filename,nsamps,nstart);
fid = fopen(filename);
fseek(fid,4 * nstart,'bof');
y = fread(fid,[2,inf],'short');
fclose(fid);
x = complex(y(1,:),y(2,: ) );

So, how can I perform Time Delay Estimation? Cross-correlation is the method to get time delay, but I still do not understand how.

Also, I am trying to look at these 2 signal in the spectrogram, but when I have images, I don't see them similar. they are supposed to be similar.

figure,
spectrogram(x_tx ,'yaxis'); title('transmitted file')
figure,
spectrogram(x_rx_on,'yaxis'); title('Received file')

I hope you can help me to understand this problem.

 

[FactChecker]

I assume that elements of the arrays x_tx and x_rx_on are values of each at the same time. There should be an entry of the correlation array that is highest positive value. That should tell you how many frames of delay there was between the transmission and the reception. Then you can convert that number of delayed frames to a time delay.

 

[Nate Duong]

I assume that elements of the arrays x_tx and x_rx_on are values of each at the same time. There should be an entry of the correlation array that is highest positive value. That should tell you how many frames of delay there was between the transmission and the reception. Then you can convert that number of delayed frames to a time delay.

for example: if I have sample rate Fs = 8MHz and number of delay frame = 800. So, time delay will be 800/8MHz =0.0001 (second) , is it right ?

 

[FactChecker]

for example: if I have sample rate Fs = 8MHz and number of delay frame = 800. So, time delay will be 800/8MHz =0.0001 (second) , is it right ?

Right. As long as the times of your two series match.

 

[Nate Duong]

Right. As long as the times of your two series match.

I also have question, If you could help me to answer.

 

[Nate Duong]

I also have question, If you could help me to answer.

The plot which is attached, show us 10 seconds receiver received from the transmitter. It means, every 1 second the receiver receives single transmit data with noise included. Can I get every single peak of cross-correlation and calculate time delay? if so, I will have 10 different delay, and I can see how much delay change in 10 seconds? for example, I will have a vector of the delay in every second [0.0018,0.0018,0.0018,0.0017,-0.001,-0.001,-0.001,-0.001,-0.‌001] is it right when I say that?

 

[Nate Duong]

Right. As long as the times of your two series match.

Here is my update code:

%% initial values:
nsamps = inf;
nstart = 0;
Fs = 8e6; % sample rate
flag = 0; % plot in the for loop

%% input files
file_tx = 'test1-qpsk-8-8000.0-ksps.dat';
file_rx_tx_on = 'tx30-rx50-2017-03-29-14.48.56-utc.dat';

%% read files
x_tx = readcplx(file_tx, nsamps,nstart);
x_rx = readcplx(file_rx, nsamps,nstart);

%% mav : maximum absolute value
mav_tx = max(abs(x_tx));
mav_rx = max(abs(x_rx));

%% condition for selected gain
if ((mav_rx >= 1e3) && (mav_rx <= 1e4))
fprintf('satisfy condition ! \n');
else
fprintf('look and adjust the gain from RX and TX !\n');
end

%% reshape in 1s window:
rx_data_time = 10; % 10 seconds data file
factor = rx_data_time/10;
matric = reshape(x_rx, [Fs/rx_data_time*factor, rx_data_time+1]);
size_of = size(matric);
len = 1:size_of(1);

% figure, plot(abs(matric)); grid on;
% figure, plot(abs(matric(:,1))); grid on;
% rx_at_1st_second= x_rx(1:len);

%% plot s1 & s2:
s1 = x_tx;
t1 = (0:length(s1)-1)/Fs;
s2 = matric(:,1);
t2 = (0:length(s2)-1)/Fs;
figure, plot(t1,s1,t2,s2); title('before xcorr'); grid on;
legend('s1 TX','s2 RX');
xlabel('Time (s)');

%% delay calculation:
s1 = x_tx;
t1 = (0:length(s1)-1)/Fs;
time_delay_matrix = zeros(1, rx_data_time+1);

for i = 1: size_of(2)
s2 = matric(:,i);
t2 = (0:length(s2)-1)/Fs;
[corr,lag] = xcorr(s1, s2);
[~,I] = max(abs(corr));
lagDiff = lag(I); % delay
timeDiff = lagDiff/Fs;
time_delay_matrix(i) = timeDiff;

if flag
figure; plot(lag,abs(corr));
title('define the delay position');
grid on;
xlabel('samples');
% a3 = gca;
% a3.XTick = sort([-30000:1000:30000 lagDiff]);

if (lagDiff < 0)
lagDiff = lagDiff * (-1);
end

s2al = s2(lagDiff+1:end);
t2al = (0:length(s2al)-1)/Fs;

figure, title('after xcorr');
subplot(2,1,1); plot(t2al,abs(s2al)); title('s_2, aligned');
grid on;
axis([0 0.025 0 2e4])
subplot(2,1,2); plot(t1, abs(s1)); title('s_1');
grid on;
axis([0 0.025 0 3.1e4])
xlabel('Time (s)')
pause(3);
end
end
%% finish
fprintf('\n Done! \n\n');

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
So, the vector time_delay_matrix (size 1x10) is the delay estimation for every 1 second window, such as:

time_delay_matrix = [ 0.0018 0.0018 0.0018 0.0018 0.0018 0.0018 0.0018 0.0018 0.0018 0.0018 0.0018]

I do not think I did right, because the variation of the delay expecting change, but my this situation in here is zero:
(0.0018 - 0.0018 = 0)

My questions are:

Am I doing right or wrong?

how much variation do I see between those 10 measurements of delay?

I also think about the standard deviation and Average, but I do not know how? and what I can do with these to see the variation of delay? in nanosecond or in millisecond?

I am computing the time delay by interpolating between samples, and compute xcorr, how slight those changes are?

 

[Nate Duong]

I assume that elements of the arrays x_tx and x_rx_on are values of each at the same time. There should be an entry of the correlation array that is highest positive value. That should tell you how many frames of delay there was between the transmission and the reception. Then you can convert that number of delayed frames to a time delay.

Hi, are you still with me?

 

[FactChecker]

Please limit your code to the part that is relevant to determining the delay and ask a specific question. I think that may be a small fraction of the code above. Also, please indent the code to show loop blocks and if-then blocks.
Why do you think the delay should change in different 1 second windows?

 

[sophiecentaur]

I am computing the time delay by interpolating between samples, and compute xcorr, how slight those changes are?

That will depend a lot on the nature of the data. If you have truly random data then there would be only one maximum with the rest of the values being zero. If the data has ' repeating' sequences, for instance. there can be spurious smaller peaks. It's a matter of the autocorrelation function.

 

[FactChecker]

If you are picking out the delay with the highest autocorrelation from a lot of samples, then that can hide a lot of occasional transient delays. One second of 8MHz data is a lot of data. Unless you think there is something that would cause a major, sustained delay, I wouldn't expect the answers to change from one second to another.

 

[Nate Duong]

Please limit your code to the part that is relevant to determining the delay and ask a specific question. I think that may be a small fraction of the code above. Also, please indent the code to show loop blocks and if-then blocks.
Why do you think the delay should change in different 1 second windows?

That will depend a lot on the nature of the data. If you have truly random data then there would be only one maximum with the rest of the values being zero. If the data has ' repeating' sequences, for instance. there can be spurious smaller peaks. It's a matter of the autocorrelation function.

the data has ' repeating' sequences for every second, the RX data is representing for 10 second and they are kind of repeating, therefore I just need to separate for 1 second, then do xcorr, the expectation will be only 1 peak of maximum value, Please look at the the attached file, do you see it's weird?

 

[Nate Duong]

If you are picking out the delay with the highest autocorrelation from a lot of samples, then that can hide a lot of occasional transient delays. One second of 8MHz data is a lot of data. Unless you think there is something that would cause a major, sustained delay, I wouldn't expect the answers to change from one second to another.

I could calculate and see every second, I have delay, such as:

delay =
0.010346125000000 0.010349000000000 0.010349875000000
0.010349000000000 0.010349750000000 0.010349750000000
0.010349750000000 0.010349750000000 0.010348875000000
0.010348875000000 0.010348875000000

the changing in every second because of noise existing.

Maybe, I am doing wrong, because when i look at the plot of the xcorr, it looks weird to me

 

[Nate Duong]

If you are picking out the delay with the highest autocorrelation from a lot of samples, then that can hide a lot of occasional transient delays. One second of 8MHz data is a lot of data. Unless you think there is something that would cause a major, sustained delay, I wouldn't expect the answers to change from one second to another.

please look at the plot.

 

[FactChecker]

the data has ' repeating' sequences for every second, the RX data is representing for 10 second and they are kind of repeating, therefore I just need to separate for 1 second, then do xcorr, the expectation will be only 1 peak of maximum value, Please look at the the attached file, do you see it's weird?

What are we looking at here?
Three time charts: tx, rx, xcorr? No, that can't be right -- the scale of the 3'rd graph Y axis is too large to be xcorr.
One second of data? No, that can't be right -- 8x10⁵ data samples at 8MHz would be 1/10 second.
I'm getting tired of trying to figure this out.