Week of 12/3

This week, we did our design presentation.  We met on Monday to begin creating the slides.  We talked to Dr. Wolff about what we should focus on for the presentation and determined what to cover: hardware, math, and software.  Each of us finished our respective slides for those specific sections.

Tuesday, we talked a little bit about additional changes for our class diagram.  We then ran through the presentation once and didn't really do much else.  Then we presented (it went a little long).  We met briefly with Dr. Wolff and got some additional advice on our class structure.

Thursday, Tyler and I worked on making the changes to the class/sequence diagrams.  We added an Image class to handle most of the math processes, taking them out of the model.  We also added an Extrinsic and Intrinsic class to store the respective matrices.  Also, I found out that using C++ methods of OpenCV, we can save and load XML files with custom data, meaning we can save multiple matrices in the same file.  This will be useful to store all Intrinsic data in one file and each respective stage's Extrinsic data in its own file.  We also changed it so that a user will specify a directory to save to/load from rather than specific filenames.  This will make it less hassle for the user.

Grady was at work today and I don't really know how much or what all he did for the project.  He mentioned that the hardware diagrams were done.

Tyler and I have been working pretty well together.

Pretty much all we need to do is pull everything together and put together the full Latex document.  We will likely meet Saturday and if needed, Tuesday.  We will have the document turned in Tuesday.

I will likely post once more next week; then we will be on Christmas break late next week until J-Term starts (Jan. 2nd).  We plan to work on the project during J-Term; hopefully we are able to make some good headway :)

Week of 11/26

This week was spent on design and preparation for the document.

Tyler and I met last Wednesday and we made a preliminary class diagram.

This Tuesday, we continued working on more design elements.  We have revised our class diagram, especially after meeting with Dr. Wolff and it is still a work in progress.  I have also done state diagrams for intrinsic and extrinsic calibration as well as scanning.  The calibration ones have changed based on MVC design and I will edit the Scanning one as well.  I feel better about our MVC design after Thursday's work and discussion with Dr. Wolff.  All in all, design is moving along fairly well.

Grady was able to do some of the hardware and hardware-related design charts as well.  He also got a book from Tosh about circuits that has a chapter about interrupts that should prove useful.

We present on Tuesday; so Tyler and I are going to meet up on Saturday (Grady will be out of town) to prepare the slides.  We will correspond with Grady to get some of the hardware information and will send the slides out so Grady can also review.

Other terms of business:

• The Vec3F structure of OpenCV does have the cross product function so we should be able to use it for our vector functionality.
• We found the fitLine function in OpenCV but were unable to find an approximate intersection function for 3D lines.  We may have to write the algorithm for this.
• Tyler looked into possible ways of integrating GUI design with OpenCV; there is a C# wrapper available but most people suggested Qt, which is likely the route we will pursue.

We should be on a good pace to supply a quality design document due in a little under 2 weeks.  We have a good design foundation and will continue making progress.

Once design is done, we will be on break and be able to come back ready to start implementation in J-Term and the spring semester.

Week of 11/19

Short week this week with Thanksgiving.  But that didn't stop us from working :)

Tuesday we met to focus mainly on design.  Tyler and I were able to work out state diagrams for intrinsic and extrinsic calibration as well as scanning.  Tyler also continued work on the document itself in other areas, such as research review and testing plan.

Grady had a major breakthrough: he got the motor spinning.  This was a great step to achieve, especially right before the holiday weekend.

We met with Dr. Wolff and discussed some things, including reviewing our requirements document briefly and discussing design, namely the classes needed.  Tyler and I came in on Wednesday and worked on figuring out the classes we will likely need as well as some methods/pseudocode for what the classes will do.  Tyler worked on the UML diagram and has a preliminary.

Overall, it was a pretty productive short week.  We will come in next week and continue working on the design document and prepare for the presentation (maybe next Saturday??) which we will give the following week.

Happy Thanksgiving to all.

Note: Here is what the plot of the red component, not intensity, looks like for the laser line:

Week of 11/12

Another week gone by...

This week was rather so-so as far as progress.  Last Thursday, Grady and I started configuring our machines.  This took a significant amount of time, but we now have 3 machines (2 Windows, 1 Ubuntu) to work with.

Tuesday, I also researched the undistort/z-component and determined we should undistort (since it uses 2D) and then create a homogeneous coordinate from the idealized coordinate.  Speaking with Dr. Wolff, he still wondered about what the undistort function did with the z-component.  So Thursday, I took the example calibration code, used the pictures provided, got the intrinsic and distortion parameters, and then plotted out the distorted and undistorted points.  It took quite some time, especially with the example code in C.  I tried to convert things to the more recent C++ openCV elements, but I ended up sticking with C (the new version of the OpenCV book comes out in December; hopefully it includes updated example code we can use).

Below is the comparison of the distorted points with the undistorted (in ideal camera coordinates).  They seem to look pretty good and I think we can go ahead and proceed using the procedure described above. (thus the z-component must be assumed to be 1).
Note: the image used is 1600x1200 and the scaling between the two is off.

On Tuesday, I also was able to use just the red component of the laser (rather than the intensity level) and the data did show a spike.  I tried in a light room as well and didn't get very good results.  More testing in a light room with better camera settings will likely occur later.  But the dark room will produce the most stark results.

Grady is still chuggin' away with the circuits, communicating with the company.  He seems to have gotten a little more of a grasp and hopefully is close to a breakthrough.

Tyler researched some on the opencv video settings.  He hasn't been able to meet recently on our normal days, so hopefully he is available more as we approach the end of the semester.

We have a design presentation coming too quickly and will likely need to turn more of our focus towards that.

UPDATE:

Here are the mappings of the corners to the distorted and undistorted images of a chessboard:

Distorted image

Undistorted

Week of 11/5

I'd call it a successful week.

Monday, I read through the entire math section in the Siggraph document. Parts of it took a while but I feel I was really able to understand almost all of it, including partially understanding the partial derivatives.  The partial derivatives are just used for minimizing to find the optimal approximate intersection between two lines.  This likely won't really need to be a focus point in our document/presentation.  Anyways, back to the other stuff; I am glad I re-read it.  In fact I wasn't even sure I ever read the coordinate system part.  It helped to show a piece we had missed as far as the inverse intrinsic matrix goes.

We met on Tuesday primarily to finish up the mathematics pipeline.  It started out fairly rough once we hit solving for lambda.  It took some time and was frustrating.  But then I looked back at the document and found a way to do it, clearing a main hitch.  From there it was fairly straightforward.  One main question remaining is the z-component of the real image coordinates.

We met with Dr. Wolff after the tech-talk and I walked through the math.  He seemed quite pleased and offered an additional piece we had missed as far as the distortion parameters went.  I feel from last Tuesday to the end of this Tuesday, at least for me, I now understand the main processes much better.  Obviously it is still in mathematical-type pseudo-code, but it is a critical foundation we can build on.

For our next meeting, we are working on a few different things, namely some of the following:

• More accuracy for red plot: taking the R value from the BGR rather than the V value from HSV
• Red plot location accuracy (coordinate rather than a pixel value); how to better determine point (left side of laser?)
• Motor/circuitry (GRADY)
• LaTeX for the math pieces (JEFF--Done Thursday 11/8)
• Settings for camera from OpenCV
• OpenCV undistortPoints method
• Using K^-1 and Distort parameters
• What about the z-component of our image?

Grady has hit a bit of a hitch in the circuit code compilation from within Visual Studio.  But hopefully he can figure it out; we have Windows installed on two machines now and will work on getting them configured more fully.

Again, Tyler is at a disadvantage at this point as he hasn't taken linear algebra or graphics, but he did read (and take notes) on at least a good chunk of the Siggraph math portion and worked on learning some basic matrix calculation procedures.

All in all, a successful week.  Things are lining up nicely.  Design document looms in the future.  And we are on a good track...

Week of 10/29

Well the requirements document is all done :)  This week we focused more on the math and the intensity plot.

Tuesday, we spent some time on the intensity plot.  I had been overcomplicating it and had been looking at the opencv histogram function.  However, we determined to pursue by looping through a row of pixels (the row is just halfway down the image) and getting the intensity.  We tried using the red frame piece we had previously generated, but the channel we were testing had only 0's.  I think the only channel is the 1st (s[0].value) but it would only contain a 0 or 1 depending on if the hsv is in the range we specified.  We then tried with hsv and got some data.  Tyler furthered it to an excel plot and Thursday we plotted the laser line in a dark room.

Tuesday was also spent in a lengthy meeting with Dr. Wolff discussing the mathematics and starting to work out an order of the operations we will be performing.  I wrote out some things Wednesday morning on our board from what we had discussed and Thursday we worked through some of it.  We were able to Skype Grady as he was working at the time.  This was a way to communicate without him actually in the room.  We struggled some (especially on how to determine the coordinates of the laser line when determining the laser plane, until finally figuring out that the line lies on a plane and thus can determine the world coordinates in that plane's coordinate system).  Tyler is at a disadvantage not having taken linear algebra, but I am trying to explain some things as we go so he isn't lost as much and can feel more included.  We still have some math to figure out, particulary the ray to a point and the ray-plane intersection to determine the coordinate in one of the coordinate systems (both the ray and the plane must be in the same coordinate system when doing the intersection).

All three of us are going to review the math portion of the Siggraph document.  I read through this already, but it has been some time ago and it will be helpful to refresh.

I think we are making headway and I hope we can keep plowing through.  It sounds like Grady has a way to move forward on the hardware side of things as well.

Week of 10/22

This week was again spent on the requirements document.  Grady and I met on Tuesday to work on it and we all met on Thursday for a few hours to wrap it up.  We updated the screenshots, finished the document including making changes based on Dr. Wolff's edit suggestions, made the Gantt chart, and made a Google Sketchup diagram of the scene we will be setting up.

It is nice to have this document done.  As far as the team, we probably should have been a little better at communicating for Tuesday as Tyler's phone ended up being dead and thus not knowing about meeting.  We should have set it up more in advance.

However, Thursday was good and we got a good amount done.

Grady was able to do some more research into the hardware as well.

All in all a pretty successful week.  Now to get back to the actual design... :)

Week of 10/15

This week was spent entirely on the presentation and requirements document.  We met on Monday to review for the presentation which was on Tuesday.  It went fairly well..kind of long because of a lot of questions.  As Dr. Wolff suggested, we should have had more pictures to help explain what was going on (which we then discussed after class, trying to nail down for ourselves how the line gives depth).

We met on Thursday to work on the requirements document.  We sent a rough draft to Dr. Wolff.  We have the following items to work on before the due date next Thursday:

• Finish up the Introduction (need to incorporate the background research we have done)
• Task breakdown
• Preliminary Timetable (Gantt chart)
• Annotated Bibliography

Other than that, we should be on our way to reaching completion of the document by the 25th.

Our team is still doing pretty good and we are ready to continue working in the second half of the semester.

Enjoy the break!

Week of 10/8 Progress

Not much to report for this week.  We met on Monday and worked some on our requirements document.  We still have quite a bit to do.  We will be presenting our requirements on Tuesday 10/16.  We are going to meet this Saturday to work on the req. doc and presentation, hopefully getting a large chunk done.

Other than that, we may also try and extend last week's progress with the red line and plot a row of pixels' red intensity.

The team is good; we all contributed input for the document and will continue to work together this weekend.

Hope to have a rough draft done by 10/18.  After the requirements are done, hopefully we can continue making progress on the design (e.g. the stage, Grady with the motor).

Week of 10/1

Since my last post, I did some research on the webcam we will be using and found that the Logitech software doesn't persist settings upon reboot.  We haven't yet tested this ourself though.

On Tuesday, I went through Chapter 11 of the OpenCV book on Camera calibration, etc. before our faculty meeting.  During our faculty meeting, we hacked through some more of the math of obtaining a plane in world coordinates.  We also were able to get the motor and laser line.

On Thursday, our group met and began testing out the laser line with the camera.  We tried to isolate the red component of the laser, but it was not very successful.  We were able to tweak some of the webcam settings and were able to get a pretty good result of isolation of the laser in a dark room (by converting the BGR image to a HSV (Hue Saturation Value) and obtaining the pixels within a certain HSV range).  The final settings will likely not be persisted upon reboot, so we will likely need to find a way to do this from OpenCV.  The webcam settings used do wash out a regular image, but the colors in the image are still clear enough to hopefully be able to be used as a color component for our 3D polygonal mesh.

We also began working on our requirements document, and Grady began researching how to inyegrate the stepper motor with software.

Our team progress this week was not ideal, but was ok.  Next week, we will continue working on the requirements document.  Grady may also continue researching the motor implementation; and we may try and figure out the settings for the webcam in OpenCV.

Personal assessment for this week is I feel I contributed a good amount.  Probably should have done more research from the Brown sites though. Team assessment: pretty good ending to an otherwise slow week.  We were able to meet for about 5 hours and work through the first look of the laser-camera interaction.

Intrinsics/distortion parameters still loom in the future...

Anyways, Go Huskies!

Week of 9/24 Progress

This week, there wasn't as much progress.

We met on Tuesday and went over what we had done the previous week. One thing mentioned for improvement was our repo.  So this week I cleaned out the repo a little removing things we didn't need. I also added an ignore file for our future commits.  I also found some additional links from Brown University that will help us out.  I went through some of the post-processing slides from the Siggraph course as well.  As for team progress, we haven't really done much this week.

Hopefully next week we can work on trying to get a checkerboard and do some research as to the camera functions within OpenCV (e.g. autofocus).  We now have an OpenCV reference book that I will try and get to next week (ch. 11 in particular).

Personal assessment is that I have contributed some additional resources for reference that includes some more info and math work.  Team assessment: not great this week as we didn't really do much teamwise outside of our mentor meeting. We probably should have met since Tuesday.  But we did check out our workspace and discussed setup a little (such as where to have our scene/camera).

Hopefully we can get the intrinsic params for the camera before too long...

Progress, week of 9/17

This week, our group met with Dr. Wolff to go into more detail on what we should review.

I spent some time on my own reading through the math of triangulation found in the Brown University/Siggraph course on building a 3D scanner. I also configured Visual Studio 2010 and OpenCV on my laptop.  I tested out code with live video and it worked.

On Thursday, our group met.  We went through the step-by-step example of the Matlab toolbox for image calibration with the checkerboard.  We also started up our website using GitHub pages.

We also have set up a GitHub repo.

This was a pretty successful week overall :)

Test Post

This is the first post to my progress blog.