Q3Map3 - Discussion and Development

[Silicone_Milk]

I will just be a little more patient

Things have come up in regards to job hunting that have left me unable to finish what I want to get done before posting the source.

Once I finish that stuff up it'll definitely get posted.

[fKd]

*bump

any news?

[Silicone_Milk]

*ahem*

Yes yes I haven't forgotten about this.

Shit had gone down that the Mavericks competitors are aware of that kind of threw everything off schedule significantly.

I just haven't had a ton of time lately to do much of anything. And the time I have had I've put towards learning to make 3D models.

[Eraser]

Ever since late March you've been saying you haven't had much time :-(

[Silicone_Milk]

Yeah well things like losing your job, living out of your car for a little while, then getting a new job and putting yourself in debt to go back to college to get a degree really makes it a bit difficult to sit down and focus on a hobby programming project :P

[Anthem]

Yeah well things like losing your job, living out of your car for a little while, then getting a new job and putting yourself in debt to go back to college to get a degree really makes it a bit difficult to sit down and focus on a hobby programming project :P

Indeed, bro. We all understand the situation. Take as much time as you need.

[^misantropia^]

What I don't understand is why you don't put the code you have so far on GitHub or Google Code or something.

[monaster]

@Silicone_Milk: My sympathies. Just put some badass boot virus in you next release to share the pain. No, seriously, life can only get better from that point, cheer up!

[Silicone_Milk]

I'm starting work on this again. That is all.

[Eraser]

\o/

[Silicone_Milk]

Made some headway today that I decided to share before heading off to work.

Here's a quick rundown of what went down after picking this project back up:
- Was rewriting the entire source in C++. Realized how retarded it was to heap up so much work for little return value. So, started from square 1 with the original q3map2 source.
- Got a git repo set up. It's private right now.
- Was able to compile the source (gtkradiant-1.6) on linux but not windows for a while. Worked on getting the source compiling on both windows 7 and linux. They're both working now.
- Fixed a buffer overflow issue I was seeing in the release build but not the debug.
- Python support is no longer working since I started from scratch. It will be added. But later.
- Not using any bindings for OpenCL. Working directly with it in C. Host code is ready for kernels to be written and executed now

Obligatory screenshot:
[lvlshot]http://www.siliconemilk.com/music/real.jpg[/lvlshot]

[fKd]

oh hell yeah! gl bro, geet er done

[Eraser]

Awesome man. I'm really anxious to see some compile time comparisons when this is done

[Silicone_Milk]

Thanks guys.

Wasn't feelin' it today but I did manage to get a little more work done.

Wrote some helper functions to load opencl functions and to deal with opencl memory buffers.

I didn't realize how much I actually liked C++ until coming back to C

[Silicone_Milk]

I've had a massive headache for the past couple of days. Not sure what's up with that but I didn't get done nearly as much as I had hoped before the new school term began.

Anyways, this thread needs another image.

Just making sure everything's loading fine before I compile a test program to run on an opencl device. What you see here is just a couple .cl files that have been loaded in to a dynamic array being printed out (just checking to make sure my code wasn't being stupid).

[lvlshot]http://www.siliconemilk.com/music/update.jpg[/lvlshot]

[Delirium]

what new features do you plan on implementing? or do you just plan on rewriting the code so its smoother?

Im actually confused, what are you intentions with q3map3? :>

[Silicone_Milk]

I'm building on the existing q3map2 code to add the ability to compile using an OpenCL Device (this can be a GPU, CPU, PhysX chip, etc.. or any combination of).

This will allow compile times to be decreased (hopefully) by quite a bit.

[Delirium]

Sweet

I'll be following progress for sure

[monaster]

Yeah, keep on working on that, I believe that could turn out really sweet (even if most of what the screenshots above show went way over my head )!

[Silicone_Milk]

Working on getting a test function working. Should have it done before my next class.

In a nutshell, the above screenshots are a -light compile with q3map2 with verbose information giving me status reports on the following:
- Does an OpenCL-enabled device exist?
- Can we use the device if it does exist?
- Can we get some memory to load the opencl functions in to?
- Are the functions we're reading to memory being read correctly?

Everything's checking out ok. Now it's getting the functions working with some of the data and creating the right results that I need to work on

Then the task of porting q3map2 code over to the gpu begins. Things'll start gettin' nasty at that point for sure.

My main issue right now is that a lot of the algorithms ("method for solving a problem" for the non-mathematically inclined) are recursive (meaning they use themselves in calculating results (ie. a factorial equation such as function(number) = number * function(number - 1) ))

The problem with this is OpenCL doesn't allow recursion. At all. So I need to rewrite the algorithms to be non-recursive which is going to take some thinking.

Disclaimer: The "factorial" recursive equation example isn't a true factorial equation and will give you gas if you try to run it.

[roughrider]

Gl. I hope to be able to use this before I turn 50

[Silicone_Milk]

Added in error reporting. Bad OpenCL kernels will cause q3map3 to fall back to using the old cpu-based code and print the build errors out to a log file.

[lvlshot]http://www.siliconemilk.com/music/error_report.jpg[/lvlshot]

[Eraser]

The problem with this is OpenCL doesn't allow recursion. At all. So I need to rewrite the algorithms to be non-recursive which is going to take some thinking.

Sounds like you're about to embark on a journey into the lands of Hurt

[^misantropia^]

What if f() calls g() and g() calls f()? Does the compiler see that as recursion?

[Silicone_Milk]

That's an excellent question, misantropia.

I will have to see. But first, I must find out why the OpenCL build log is suddenly spewing nonsense as if I'm outputting random locations in memory and not from the buffer I've *been* outputting from.

The thing that's making me scratch my head is that rolling back a revision to when the code produced the above screenshot doesn't fix the problem.

We'll see what Valgrind has to say about these shenanigans.

EDIT: Rather embarassing, but I figured it out without valgrind (too lazy to add the new ocl libraries to scons to build a linux version of the program....). It turns out the error was a rather stupid one - double-dereferencing

Fixed that minor issue and the error logs work once more. I just wonder when I added that in to the program

Anyways, I have a new goal for tonight/tomorrow morning - .cl file parser to automatically allocate memory for kernel arguments.

When you write a kernel for OpenCL you also need to allocate a buffer of memory for EACH argument you want to pass in to the function. It's a tedious task to have to do it every time you write a new function so I'm going to make the program do it for me.... after I upload an image of a test kernel running in q3map3 using hard-coded arguments

Progress is good at least!