Spring break this week, but that didn't stop us from working :)
Grady was having some problems integrating the serial code into the main application. He debugged for hours and couldn't figure it out. Tyler and I met on Thursday, and he basically spent the whole time trying to figure out a linker error. We finally were able to get it working. What it was was a LPCTSTR error. The serial code, which is being used as a lib, uses a LPCTSTR type in some of its methods. However, when it gets compiled, it is converted to an unsigned (const?) char *. So when using the lib, the main app only references the header file, which contains the LPCTSTR type. When trying to use one of the methods with that type as a parameter, it causes a linker error for an unresolved external for the method signature. To fix this, we changed the LPCTSTR to the unsigned char in the serial code's header and cpp file and instead of using a __t() macro, we pass in a string directly when using the code in the main app. This fixed it (finally).
On another note, I continued trying to debug the scanning. I did fix a bug for when no red was found in an image. But I still haven't figured out what's causing the non-perpendicular traits of the planes. I have been converting to world coords and comparing to the origin of the back plane (which is the translation vector). It seemed that the camera-coord z-coord may have been off by 1 unit in one of the executions. I'll look into that more next week. Hopefully I can find something quickly!
Until next week!
Thursday, March 28, 2013
Friday, March 22, 2013
Week of 3/18
Scanning has begun, but there is still plenty of work to be done.
Since my last post, I was able to refactor a lot of the main scanning code to break up where the for loops are. This provides for cleaner code. I also set up the overlay view to be able to test the scan. When running through the scan (with just video of myself -- no laser), I got some memory exceptions. It turns out you cannot multiply a float matrix (32F) by a double matrix (64F), so I fixed the errors and was able to run through a scan with no errors.
On Tuesday, Tyler and I came up with a makeshift scanning stage idea. I put it together and attempted a scan...the results were definitely not good. Wednesday, I came back and started working through it and determined there was some logic errors in our red midpoint calculation. One error was when checking for a zero crossing, I had a check if it was equal to zero...well if it is black, the midpoint is zero (max and min red components are zero). So I took out this piece (at least for now). Also, our idea of calculating the midpoint at each pixel was causing some problems. If a black pixel had a min red point of 0 and a max of 1 for a midpoint of 0.5, it was finding a zero crossing between a -0.5 and a 0.5 value. This means that if we find the midpoint at each pixel, every pixel must be hit by the laser (as the Brown/Siggraph notes mention for the shadow). Instead, I changed it to find the midpoint across a row of pixels. When the lights are off, this should work fine. For the Siggraph structured light shadow, it probably wouldn't as the object itself causes a shadow. But when red is the only visible piece, it is okay if not every pixel is hit by the laser.
When I made that change, this is what resulted as the scan of a coffee cup, shown in a program called MeshLab:
Since my last post, I was able to refactor a lot of the main scanning code to break up where the for loops are. This provides for cleaner code. I also set up the overlay view to be able to test the scan. When running through the scan (with just video of myself -- no laser), I got some memory exceptions. It turns out you cannot multiply a float matrix (32F) by a double matrix (64F), so I fixed the errors and was able to run through a scan with no errors.
On Tuesday, Tyler and I came up with a makeshift scanning stage idea. I put it together and attempted a scan...the results were definitely not good. Wednesday, I came back and started working through it and determined there was some logic errors in our red midpoint calculation. One error was when checking for a zero crossing, I had a check if it was equal to zero...well if it is black, the midpoint is zero (max and min red components are zero). So I took out this piece (at least for now). Also, our idea of calculating the midpoint at each pixel was causing some problems. If a black pixel had a min red point of 0 and a max of 1 for a midpoint of 0.5, it was finding a zero crossing between a -0.5 and a 0.5 value. This means that if we find the midpoint at each pixel, every pixel must be hit by the laser (as the Brown/Siggraph notes mention for the shadow). Instead, I changed it to find the midpoint across a row of pixels. When the lights are off, this should work fine. For the Siggraph structured light shadow, it probably wouldn't as the object itself causes a shadow. But when red is the only visible piece, it is okay if not every pixel is hit by the laser.
When I made that change, this is what resulted as the scan of a coffee cup, shown in a program called MeshLab:
This is actually an inverted view. Since we are converting to the back world coordinates, this is a view from behind. So if we want to plot the view from the front, we will probably want to rotate around the Y-axis by 180 degrees.
The next problem is the depth. This scan isn't really that great depth-wise. It is more of a 2D image with a coffee cup sticking out (in from this view). The stage pieces are not perpendicular but more of 130 degree angles. I spent 5+ hours thinking/debugging this, including scanning just a our stage. I thought it might be how we are obtaining the extrinsic values...but I'm not convinced of this. I looked at the Siggraph notes, but I'm still not sure what the problem is. Does it really matter where the checkerboard is on the stage when you do the extrinsic calculation? The lambda value is n^t(plane_point-camera_origin)/(n^t(camera_point))...so do our assumptions of (0,0,0) as the plane origin and (0,1,0) [or (0,0,1)] as the plane normal work? I want to talk to Dr. Wolff about this problem as so far I haven't been able to find a problem.
Meanwhile, Tyler has worked on fixing some things in the overlayview and scanning and preparing for the progress bar. He changed the overlayview to use an alpha channel.
Grady got the interrupt to work correctly and has a serial library written. He is working on integrating it into the app. He also worked with a DC motor instead...but we might just stick with the stepper motor if we can get a smoother motion.
During spring break, we may meet once or twice. We need to get our scanning stage built. After spring break, crunch time gets closer. We need to figure out the progress bar and figure out the scan problems and getting the stepper motor integrated. The scan problems are what concerns me most as it is very difficult to debug logic errors with it. But hopefully I can find what is wrong and/or get some advice/help from Dr. Wolff.
Spring 'break' is here.
Thursday, March 14, 2013
Week of 3/11
A week closer to a working 3D Scanner.
This week, as a team we didn't do much since we had class on Tuesday (and then a meeting with Dr. Wolff), and on Wednesday we prepared for our presentation on Thursday. We presented about our progress on the project and covered the GUI, calibration, and the hardware. I showed a remapping of found points on an image, and Tyler was able to implement a rough version of the region-clicking GUI.
However, since my last post, I was able to implement the red-component finding, laser-plane finding, and laser-object intersection finding. In the red-component, I determined that we can loop through each row in the object region as well (just like the back and ground planes) and do the same interpolation to determine the x-subpixel location. Based on where in the imageNum loop, we know which frame that particular subpixel value is found and can associate it with the laser plane at the same image. The interpolation used is a basic linear interpolation: x = x_0 + (0-delta(x_0))/(delta(x_1)-(delta(x_0)).
In the laser plane detection, I basically implemented the algorithm described in our design doc and shown in Taubin's notes (such as the approximate intersection). I also used cv functions such as undistort and the homogeneous conversion function.
The laser-object intersection was also basically just an implementation of the algorithm stated in the design document.
These things have yet to be tested, so hopefully in the next week or so, we can get to the point of performing a basic scan of some sort to: test the red-component and then find the plane. The object-intersection testing will likely come a bit later.
Tyler will work on updating the GUI more and Grady will work on figuring out why the interrupt is going twice and then branch off the repo to begin work on integrating it into the application.
I did some research on polygon rebuilding. We will likely want to try and output our data as a VRML file to use in another program (such as an old version of Blender {maybe} or Taubin's Java program) if we don't use OpenGL to build a polygonal mesh (which I think could be difficult and unattainable with our schedule).
Until next week...
This week, as a team we didn't do much since we had class on Tuesday (and then a meeting with Dr. Wolff), and on Wednesday we prepared for our presentation on Thursday. We presented about our progress on the project and covered the GUI, calibration, and the hardware. I showed a remapping of found points on an image, and Tyler was able to implement a rough version of the region-clicking GUI.
However, since my last post, I was able to implement the red-component finding, laser-plane finding, and laser-object intersection finding. In the red-component, I determined that we can loop through each row in the object region as well (just like the back and ground planes) and do the same interpolation to determine the x-subpixel location. Based on where in the imageNum loop, we know which frame that particular subpixel value is found and can associate it with the laser plane at the same image. The interpolation used is a basic linear interpolation: x = x_0 + (0-delta(x_0))/(delta(x_1)-(delta(x_0)).
In the laser plane detection, I basically implemented the algorithm described in our design doc and shown in Taubin's notes (such as the approximate intersection). I also used cv functions such as undistort and the homogeneous conversion function.
The laser-object intersection was also basically just an implementation of the algorithm stated in the design document.
These things have yet to be tested, so hopefully in the next week or so, we can get to the point of performing a basic scan of some sort to: test the red-component and then find the plane. The object-intersection testing will likely come a bit later.
Tyler will work on updating the GUI more and Grady will work on figuring out why the interrupt is going twice and then branch off the repo to begin work on integrating it into the application.
I did some research on polygon rebuilding. We will likely want to try and output our data as a VRML file to use in another program (such as an old version of Blender {maybe} or Taubin's Java program) if we don't use OpenGL to build a polygonal mesh (which I think could be difficult and unattainable with our schedule).
Until next week...
Friday, March 8, 2013
Week of 3/4
Back again for another week :)
The core of intrinsic and extrinsic calibration is done, and I can save and load the matrices!
This week, we had a major hardware breakthrough: the interrupts are getting triggered! Now there is still plenty of work for Grady to do with it, but this was a major hangup that is great to get past.
On the software side of things, this week I spent quite a few hours studying/attempting to understand/thinking about the red component and how we will get the left edge of the laser according to the Brown notes. They find the midpoint value (for them it is intensity, for us the red component) per pixel over all frames and then compute the difference image for each image. Then to find the subpixel location of the laser line in a specific frame, you find where the difference image crosses 0 in a specific row of pixels and interpolate to that point based on the pixels before and after. You take all subpixels for a region and find the best fit line. Then for finding when the laser hit a particular pixel on the object, you loop over a particular pixel over all frames and find the zero-crossing; I made the assumption to choose the left side of the zero-crossing as we will be dealing with the left side of the laser line.
We discussed it with Dr. Wolff and since he trusted the document, we went with it. I began implementing it on Thursday. Dr. Wolff mentioned a potential issue of non-negative numbers being used for the channels in an image (the image we get from the camera is a 3 channel matrix of unsigned chars between 0 and 255 inclusive) since the difference image needs to be allowed to be negative. Uchars, schars, and chars don't provide the range we may need, so I convert the red channel matrix from CV_8UC1 to CV_32F (floats) to be able to use negative value.
In addition, this week involved some more plumbing, including setting up some more MVC, adding OverlayView classes, and some message displaying/error handling (e.g. if no camera is connected).
We are planning on working on Saturday and hopefully we can knock out more of the red component stuff and get the OverlayView working (and hopefully Grady can make more progress on the interrupts).
We present our first technical presentation of the semester and plan on talking about displaying an cv-produced image in Qt (and maybe some other info about Qt) as well as what fixed the interrupt issue (hopefully to a level the audience can at least partially understand).
Until next time...
The core of intrinsic and extrinsic calibration is done, and I can save and load the matrices!
This week, we had a major hardware breakthrough: the interrupts are getting triggered! Now there is still plenty of work for Grady to do with it, but this was a major hangup that is great to get past.
On the software side of things, this week I spent quite a few hours studying/attempting to understand/thinking about the red component and how we will get the left edge of the laser according to the Brown notes. They find the midpoint value (for them it is intensity, for us the red component) per pixel over all frames and then compute the difference image for each image. Then to find the subpixel location of the laser line in a specific frame, you find where the difference image crosses 0 in a specific row of pixels and interpolate to that point based on the pixels before and after. You take all subpixels for a region and find the best fit line. Then for finding when the laser hit a particular pixel on the object, you loop over a particular pixel over all frames and find the zero-crossing; I made the assumption to choose the left side of the zero-crossing as we will be dealing with the left side of the laser line.
We discussed it with Dr. Wolff and since he trusted the document, we went with it. I began implementing it on Thursday. Dr. Wolff mentioned a potential issue of non-negative numbers being used for the channels in an image (the image we get from the camera is a 3 channel matrix of unsigned chars between 0 and 255 inclusive) since the difference image needs to be allowed to be negative. Uchars, schars, and chars don't provide the range we may need, so I convert the red channel matrix from CV_8UC1 to CV_32F (floats) to be able to use negative value.
In addition, this week involved some more plumbing, including setting up some more MVC, adding OverlayView classes, and some message displaying/error handling (e.g. if no camera is connected).
We are planning on working on Saturday and hopefully we can knock out more of the red component stuff and get the OverlayView working (and hopefully Grady can make more progress on the interrupts).
We present our first technical presentation of the semester and plan on talking about displaying an cv-produced image in Qt (and maybe some other info about Qt) as well as what fixed the interrupt issue (hopefully to a level the audience can at least partially understand).
Until next time...
Subscribe to:
Posts (Atom)