Why is my Stereo Vision Live Stream Displaying a Black Box?

In summary: Additionally, the disparity map may need to be adjusted to <code>disp = disparity(J1, J2, 'DisparityRange', [0, 112])</code> in order to properly reconstruct the 3D scene. It is also important to note that the code will continue to run until the figure window is closed. To fix this, you can add a condition to break the loop after a certain amount of time or when the figure window is closed. In summary, there are several possible issues with your code that may be causing the block box without any live video.
  • #1
PainterGuy
940
69
I'm trying to perform depth estimation on stereo vision using live feed. I'm using two webcams. The code below doesn't give any error but I only get a block box without any sign of live video! Could you please give it a look?

1619426762961.png
Matlab: 
leftCam = imaq.VideoDevice('winvideo', 1,    'MJPG_320x240');
rightCam = imaq.VideoDevice('winvideo', 3, 'MJPG_320x240');
leftCam.ReturnedDataType = 'uint8';
leftCam.DeviceProperties.WhiteBalanceMode = 'manual';
rightCam.ReturnedDataType = 'uint8';
rightCam.DeviceProperties.WhiteBalanceMode = 'manual';

if ~exist('stereoParams','var')
load stereocalibration
end

% Run 3D image reconstruction on live video
ax = axes;
maxDepth = 5 ;

while true

imageLeft = step(rightCam);
imageRight = step(leftCam);

% Rectify
[J1, J2] = rectifyStereoImages(imageLeft, imageRight, stereoParams, 'OutputView','Full');

J1 = rgb2gray(J1);
J2 = rgb2gray(J2);

%Compute Disparity Map
disp = disparity(J1, J2)%, 'DisparityRange', [0, 112]);

% Reconstruct 3D scene from Disparity Map
pointCloud = reconstructScene(disp, stereoParams)./1000;
% Remove all points greater then maxDepth m away from the camera
z = pointCloud(:,:,3);
z(z < 0) = NaN;
z(z > maxDepth) = NaN;
pointCloud(:,:,3) = z;
% Continue running until figure window is closed
if ~ishandle(ax)
    break;
else

pcshow(pointCloud, J1,    'VerticalAxis',    'Y','VerticalAxisDir',    'Down',    'Parent', ax);

xlabel('X (m)');
ylabel('Y (m)');
zlabel('Z (m)');
xlim(ax, [-.8,    .8]); %-0.8 to 8
ylim(ax, [-.8, .8]);    %-0.8 to 8
zlim(ax, [0, maxDepth]);
daspect(ax,'manual');
pbaspect(ax,'manual');
drawnow;
end
 endrelease(leftCam)
release(rightCam)
 
Physics news on Phys.org
  • #2
It looks like there is an issue with your code. Specifically, the line <code>imageLeft = step(rightCam)</code> and <code>imageRight = step(leftCam)</code> need to be switched so that imageLeft is coming from the leftCam and imageRight is coming from the rightCam.
 

1. What is stereo vision and how does it work?

Stereo vision is the ability of the brain to perceive depth and dimension from the images captured by both eyes. This is achieved through the process of binocular disparity, where the brain compares the differences in the images from each eye to create a 3D perception of the world.

2. How is stereo vision used in live streaming?

Stereo vision is used in live streaming by using two cameras to capture the scene from slightly different angles, mimicking the way human eyes see. These two video streams are then combined and transmitted in real-time, providing a more immersive and realistic viewing experience for the audience.

3. What are the benefits of using stereo vision in live streaming?

Using stereo vision in live streaming can provide a more realistic and immersive experience for the audience, as it allows them to perceive depth and dimension in the video. This can be especially beneficial for live events such as concerts or sports, where viewers want to feel like they are part of the action.

4. Are there any challenges in implementing stereo vision for live streaming?

One of the main challenges in implementing stereo vision for live streaming is the need for specialized equipment, such as cameras and software, to capture and process the two video streams simultaneously. Additionally, the synchronization of the two streams can also be a challenge, as any delay or mismatch can affect the overall viewing experience.

5. What other applications can stereo vision have besides live streaming?

Stereo vision has a wide range of applications, such as in robotics, virtual and augmented reality, and medical imaging. It can also be used in autonomous vehicles, as it allows for a more accurate perception of the environment and better decision-making capabilities.

Hot Threads

Recent Insights

Back
Top