Okay, to be honest,I only got about 20% of that, but something that can tell pony and non-pony images apart sounds awesome.

So, shine on you technologically advanced diamond.

>But am I crazy? Is this relevant to any other bronies?

You are only crazy in thinking that this could be irrelevant! This has a whole slew of potential applications. For example, it is possible to identify the concentration of ponies in other online forums, website, etc. Whether this is to improve their pony concentration or encourage existing pony submissions is up to the user. With every high accuracy, it may even be possible to identify webcomics that have featured ponies (something that is currently done manually and reported on EqD). The potential is endless!

Looking at http://callstack.org/svn/Experiments/PonyMatch/train/, I have to ask: why do you use such a small dataset? Or did I misunderstand and this is the set you are using for validation? Or is this just an example and your actual training data is much larger (I mean just ponychan alone has so many images).

My AI is a bit rusty so this might be complete gibberish, but I don't understand why you have half a million inputs for your neural net. Shouldn't you do some kind of feature extraction and selection beforehand? (Like what you are currently doing, but for the original problem.)

I would have thought that binary classification would be easier than distinguishing between multiple ponies. (But if I recall correct, neural net behaviours are a bit unpredictable.)

Also, have you tried (looking for and) using different libraries. I mean image classification (especially binary classification) must lend itself pretty well to "generic" algorithms (although not necessarily with good results).

Anyway, you seem to know what you are doing and I'm just thinking out loud here.

Thanks for the reply!

Those are all really neat ideas, and I appreciate the encouragement to know this may have some use outside of what I originally envisioned. I just hope I can get it accurate/reliable enough.

You are correct, the dataset is really small. And it is my current, complete training data. So this first round is not going to be very thorough.

There are two reasons why the set is so small: The first, and biggest reason, is limitations of time and processing power. When I first slapped together my NP-complete training script, I was using a training set of only twelve pictures, but it was set to take five and a half days to complete. I toned down the variance a bit, and right now I'm having it chug through a four day training session on just those 35 pictures. If I double the number of pictures, it's going to double the time, and it will take more than a week to get some results. This will be fixed when I have a suitable genetic optimization training algorithm in place (hopefully.) The other reason is that I don't have a whole lot of suitable pony pictures for this first round of training. This is an area I wasn't sure if the brony community could lend any assistance or suggestions. For this first round of training, I'm exclusively looking for clear screen shots of the show, of vivid saturation, and I don't have many. The next stage of training would involve throwing in desaturated (or otherwise low quality) screen shots, fan art, animated gifs, and transparent png's (which I do have a lot of.) The third stage of training will involve throwing in a wide range of non-pony pictures into the mix. In each stage I'll be paying close attention to accuracy, and which directions the settings are going. And with each round, I'll be looking for ways I can change pony-net to compensate for the increasingly diverse input.

Well. I had a quarter million inputs because (accidentally said half million in OP in a brain fart moment)... my approach was probably rather naive. I used magickcore to convert any kind of image to a 500x500 pixel image. It would scale proportionately, and fill in any horizontal or vertical left over space with #FF1234. Likewise, it would fill in the background of any transparent image with #FF1234. And when it came to animated gifs, I just had it take the middle frame. But this allowed me to convert any/every image into an array of 250,000 integers, that I sort of just... fed into a neural net. I wasn't exactly expecting it to blossom into HAL, but I was hoping with enough data and training that I could get it to pick up on such patterns as: #FF1234 should be ignored, pony pictures correlate to images with contiguous regions of high saturation and bright colors, and any image with the complex gradients and noises of photographs is not a pony picture. So I would have been fairly pleased if I could have trained it to just tell the difference between cartoon images and photographs. If I got that far, and just made that proof of concept, I was going to start massaging the data and running some image processing before-hand to try to achieve something more useful. But I never made it that far.

The neural net was just going to be binary classification. It's this new color matching approach that first tried looking for specific ponies. Binary classification fulfills the intended role more completely, but pony matching could end up fulfilling the same role to a lesser extent while still having more applications. (Not to mention being easier to pull off.) Ideally I would have both. Binary classification to determine if ponies exist in the image, and subsequent matching algorithms to find out which ones are there. (Pony-Net is organized with this idea in mind. That kind of logic would reside in pinkie-sense.h, which currently does little more than call pony-scan.)

I have tried looking for libraries, but have not been met with much success. I thought this would be a generic problem, with a lot of generic solutions, but finding them has proven difficult. Most of what I have found has only questionable relevancy, is commercial, and would cost a whole lot more to license than I could afford for a project like this.

I'm sorry if I gave the impression that I know what I'm doing. I didn't mean to trick you like that. :)

But I really appreciate your thought-out-loud thoughts and meaningful feedback.

Well, for your entertainment, here's some more thought after some random googling.

From the second answer of this
http://stackoverflow.com/questions/3906682/image-classification-in-python
it seems that we probably need to address the feature extraction problem relatively early. Although, feeding the neural net with "raw pixel" data as you have done was worth trying just to see what it gives.

That link also references a library (OpenCV) which seems like it could be useful to extract features. Maybe it would be worth trying to use that library to get a set of random features (probably whichever is easiest to implement and get working first). I don't know how OpenCV compares to imagemagick/magickcore.

Then do some kind of parameter selection. Presumably, this will give you an idea of which features are useful. (You can then keep the important features for your neural net.) There seems to be quite a few implementations of Random Forest suggested by the first answer of that link.
(see for example the end of http://en.wikipedia.org/wiki/Random_forest). Hopefully, one of them does not require too much work on your part to format the input/output.

On a side note, R is a popular statistics program and there is probably a package for your distribution (just in case you didn't already know).

This quote from that link seems a bit worrisome though.
> The solution may seem over-engineered but machine learning has never been a simple task even when the difference between pages is simply that they are the left-hand and right-hand pages of a book.
but maybe they are aiming for higher accuracy that what you are.

Also, since you've mentioned distinguishing between photographs and cartoon images, there is this
http://stackoverflow.com/questions/1518305/image-classification-detecting-an-image-is-cartoon-like
whose solution also seem to be using imagemagick (except it uses shell scripts rather than the C libraries). The idea surprised me a bit, but given the outcome, it might be worth a try.

As for your initial training script, can you explain to me what the algorithmic problem is and what is your current approach (like how you explained your approach with the neural net with large input). I could read the code but I think its better if I can get your interpretation. Maybe I can make some suggestions to improve your running time (that are not too difficult to implement).

I have to say that I'm a bit concerned about the small dataset because of everything I've been told about overfitting.

I don't have any good quality images but, as you've suggested, it may be worthwhile asking on this board. But I think that somepony are getting confused by all the terminology here.
>>12346
Maybe you want to start a new thread just asking for clear screen shots of the show, of vivid saturation.

Interesting, I've been working on a similar project as well :) My goal is to have images automatically sorted depending on whether the image is pony, and if yes, what ponies are in it. I'm way too lazy to categorize the images by hand.

What I'm doing is running an edge detection filter on the image and then computing the average noise-per-pixel ratio. Cartoony images have a lot less noise than real-life ones, so this gives pretty good results. I still have to find a good threshold to classify the images correctly, but fortunately I have a 12k pony folder, which will make finding a good average a lot easier :)

To find out which ponies are present in the image, the program first identifies areas of roughly the same color and estimates their respective importance based on their size. Then the main palette colors of the image are calculated based on that. Every pony has their own color scheme, so right now all it does is match the different pony palettes to the colors present in the image to check whether the pony is there or not.

While this works better than expected, this is of course a very kludgy approach. Backgrounds for example attribute a lot to the colors of the image, even though they don't have anything to do with the actual content of the image. An even worse situation are crowds of ponies, where pretty much every color used for ponies in the show is present, even though some ponies are missing (for example, an image with Rainbow Dash and Rarity makes the program think that Trixie is there too, because her main colors are present in the image).

What I'm going for now is first approximating the shape of each of the detected uni-colored areas using least-squares (probably with rotated ellipses/rectangles), and then calculating the proximity of the areas to each other. If a pony color palette matches the main colors of the image, it would then check whether the respective colors are actually close to each other.
What I'm struggling with now is the obscene runtime of this. To approximate the shape of an area correctly, I need to include most of the boundary points in the least squares algorithm. That, on the other hand, yields a matrix with a few thousand entries. And computing the QR-decomposition of that is just very expensive.

So yeah, I doubt that I'll have any success with that, but on the off-chance that this actually works, I'll let you know.

The first batch of training is complete, and I now have a demo of its result. (Please don't spread it around much. Both the website and current state of pony-net are terribly embarassing.)

Demo: http://callstack.org/pony-net/

Canterlot:
Thanks for all of that information! It looks like there's a lot of cool and useful leads there. I haven't had a chance to thoroughly investigate all of it, but I've at least given every bit a glance-over.

The noise/smoothness is going to get (mostly) implemented in pony-scan.c as one of the next upcoming changes. It's something I have been mulling around in my head for a little while, and test data has shown it's going to be quite necessary. I have a few different ideas about how I want to do this.

I'll definitely be giving opencv a thorough investigation. Once I'm satisfied or flustered enough with the current color matching algorithm, (thanks to you) I'll be looking into discerning features. If that works well, it will serve to not only help the cause of pony/non-pony differentiation, but it will let me feed the color scanning algorithm data exclusively from detected pony features.

The initial training script is as naive as it gets. It has a set of all the pony-match arguments, along with their min/max and incriment values. Then from that set, it creates the cartesian product of all possible combinations of all arguments across the span of their min and max values. Yeah. It's time intensive.

But the old is out, and the new is in! I finished my new training script based on genetic algorithms, and it's working beautifully. I'm getting good results in seriously large fractions of the time. This means I'm now free to boost the size of the training set. Which... is necessary. You are correct, my current set sucks. And it really, really shows. Perhaps the biggest problem I'm seeing is that pony-net is currently favoring Twilight Sparkle considerably, because she showed up the most out of any character in my training data. (I also should change a few of her secondary colors into outline colors.) But my new training set will need to have a balanced amount of all the characters.

Hmmm, starting a new thread to ask sounds like a good idea. Do you know what board might be the most appropriate place to post it?


Thunabrain:
Your take on this problem sounds really cool so far! Yeah, it's looking like noise/pixel ratio is going to play a good part in my initial pony/nonpony differentiation. As I'm learning just now, finding good thresholds is a really big and difficult part of these sorts of problems. I'm thrilled so far though by the genetic optimization solution I cobbled together; I would definitely suggest giving it a try if you're having troubles.

I definitely experienced the problem you mentioned with backgrounds and multiple ponies. It's what prompted me to revise the algorithm to look contiguous colors, instead of the presence of colors anywhere in the image.

Rainbow Dash + Rarity = Trixie. Dang. That's a kind of a problem I have yet to come up against, because I have only defined the mane six. I can see where there will be a lot of false positives when I start adding in all side characters. :(

"approximating the shape of each of the detected uni-colored areas using least-squares" I can generally follow what you're saying, but dang. I'd have to do some research to figure out how to do that m'self. This approach sounds really cool, but I can definitely see where it could become processor intensive. That's a fear I have. I'm worried that the more I throw at pony-net, the longer it's going to take to check each image.

Confound the limitations of these processors. I want cheap, disposable super computers from Walmart. And where the heck are these future quantum computing processors when you need them? I want the future. Now.

I would love to hear about updates on your project. Especially any new difficulties you come across, and what ways in which you've approached them. Also, I don't know what language you've written your program in, but you're free to take any bit of pony-net code if it's compatible (and if pony-net ever matures enough to the point of being useful.)

Interesting idea! I never could get my own neural net or genetic algorithms to work worth beans, so the fact that you're managing to beat the naive approach is awesome!

On the topic of image processing and neural nets, I think you may be interested in reading Wikipedia's article on the retina. There are a few layers of interconnected neurons connected to the photoreceptors which serve to reduce and simplify the data even before it goes down the optic nerve to the brain. Interestingly, the processing is very similar to edge detection!

Also, though not ponies, I remember reading this article on CodeProject a few years back (http://www.codeproject.com/KB/library/NeuralNetRecognition.aspx) In this case, they're making a neural net which recognizes handwritten digits. It may be an interesting read :)

>>12632
Thanks! I did a few test and the results seem decent.

I took a look at OpenCV myself (didn't originally intend to but it looked interesting) and it looks like the documentation on the library is a bit lacking. I mean, there is a fair amount of documentation, but very few tells you what functions do or how to use them in combination (except for a few predefined tasks).

I've yet to figure out which part is good for extracting some features (say as a fixed array of integers for example). The library has some built in machine learning functions but the examples that use them don't say how the input data is generated (actually, all their sample data are not computer vision related). Hopefully, you will have better luck when you get to take a look.

Glad to hear that the new training script is working well. I just joined this board recently I'm quite clueless about where to ask for images. But if /meta/24783 is accurate, probably /pic/ is the best place.

>>12431
This is probably a silly question, since there is less noise on edge detection, would it be possible to take the edges as boundaries, use some kind of flood fill to get regions (connected components) and then consider these regions as ponies if they are sufficiently large and uniform in colour. And then use the regions found this way to determine overlaps.

Least square on the other hand is too popular of a problem for me to know how to improve on it. Are there (faster) ways to solve get an approximate solution instead? I mean you are losing precision from other sources already, so the trade-off might be worth it (but I don't know any algorithms).

On a side note, OpenCV (which I still have trouble using properly) has a fitEllipse function which seems to run at a good speed. I'm not sure if it does what you want, though.

It will be interesting to see how well this approach does.

>>12632
For super-computing power, you can probably just get yourself a large number of old computers and put them in a local network. I think most tasks here can be done in parallel.

I heard that they can get some weak (few qubits) quantum computer to exist (and do computation) for some fractions of seconds. It also seems like it would be hard for somepony used to classical computers such as myself to come up with good algorithms for them.

It looks like most of my previous post doesn't make much sense, partly due to my lack of understanding of computer vision (CV).

First, "feature" means slightly different things in AI and CV. In CV, it means distinguishing characteristic or point of interest, which depends on the image (for example, its corners, contours, etc). In AI, it means some global property which all image have a value for, such as having ponies or not (which what I thought it meant all along).

So OpenCV is designed to extract features of the CV type. And then we'd still have to manually fiddle with it (determine how to turn them into features of the AI type), which is what you two have been doing all along (such as using your prior knowledge about cartoons and pony colours). This is probably the best way now anyway.

In short, if this post is too long, you should probably keep doing whatever it is you are already doing (and only look at other tools if you want to compute something specific about the image that you had in mind).

By "feature extraction", I suppose what I really meant was some kind of principal component analysis (PCA), which according to the internets, works poorly on images (it is prone to errors under translation, rotation, etc). A possibly better way is to use techniques called SIFT (Scale-invariant feature transform) and SURF (Speeded Up Robust Feature) from CV but these gives me a set of vectors of fixed length for each image. The number of vectors vary depending on the image and parameters given to the algorithm. Each vector is also attributed to a specific xy point on the image. Apparently, the closer two vectors are, the more they represent a similarity in the two images.

So, initially, it seems likes there's no direct way to compare the vectors of multiple images, since the number of vectors vary (it is possible to compare two images using some kind of nearest neighbour search to "match" the vectors, but I'm not sure what is the right way to extract a "certainty" value from the end result (and depending on how accurate we want the result, may take quite a bit of computation).

One possible way to compare multiple images is to compute the vectors for all images, cluster these vector into a set of k "central" vectors (the number k depends on the number of final AI features you want), and then label each image by a length k array whose ith entry are the number of vectors of the image close to the ith central vector. This is referred to as the "bag of words" model (and this confused me at first because it sounds like something used for text rather than images). There is a demo of this at
http://people.csail.mit.edu/fergus/iccv2005/bagwords.html
(I haven't tried downloading it yet).

Second, what I said about flood fills seems to be essentially what the first post already described.
>>12343
Oops. Although, the "scanning lines" confuses me a bit.

WHAT THE FUCK I'M WORKING ON THE EXACT SAME THING

After reading through the thread, here's my progress: I'm trying to write a pony detector in C# by looking for coloured regions just like you. I've kinda got a basic algorithm working, but I'm kinda stuck trying to implement good shadow removal.

I have no prior education in CV and I'm going by http://szeliski.org/Book/ learning as I go.

Anyway, how about we get a collaboration going? Or should I find something else to do?

>>13059
Nice. I'm surprised at how many people are trying this. It looks like you know much more than me about CV then.

Unless I'm mistaken, from your image, it looks like you are trying to recognize pony toys in a picture. I believe that (current) Roobles is trying to recognize ponies in the cartoon (in screenshots). (Either by distinguishing between cartoons and photographs or identifying ponies in a cartoon image).

From my (very) limited knowledge of CV, I'd say ponies from a photograph might be easier to recognize. I would even suggest keeping the shadows to help recognition! In fact, I think that many "standard" method might have trouble in the cartoon setting because there are no lights (which tend to create some kind of identifiable gradient). But I haven't tried it myself so this is just a guess.

It seems that, to recognize object from real life, you can use something called "Haar wavelet" to track "Haar-like features" (I haven't read their definitions yet, I just know this method is used in object detection). It is possible to train a classifier to recognize an object of interest to you.

I found this tutorial which seems ok
http://note.sonots.com/SciSoftware/haartraining.html
(in my opinion, there are many "bad" tutorials out there, even this one isn't as good as it could be). Its a bit OpenCV specific (which is what I was looking for) but maybe there's better if you just want the theory.

Other methods of possible interest are SIFT and SURF from my last post (but again, I haven't read up on their details, only what (vaguely) they are supposed to do). You might also want to look up "Histogram of oriented gradients" (which is apparently new and used for people detect in the sample code from OpenCV).

The image I'm posting can be obtained more or less from the sample programs that come with OpenCV. Regions are easier to identify in cartoon images (I think). Unfortunately, I don't have a lot of code to provide (mostly running sample code). Originally, I joined this thread because it looked like a very interesting idea. I didn't intend to get too involved (not even to my current extent).

On the other hand, Roobles' code is available (and you can try the web interface first just to see what it does).

Thunabrain hasn't gotten back to us yet so I don't know anything about code.

Bump for science.

Guuuuuuuyyyyyyyyys!

So, I was going to take a look at Roobles' code today but it looks like the website (in the first post) is down. Does anypony have a copy?

Ghhh.

Anyway, like I said, since we have a common goal, we would be more effective if we coordinated our efforts. Those interested, email me at [email protected] .

Sorry for not responding in so long! I had a lot of RL stuff to deal with.

The only thing that I made progress with in this time is the realization that computer vision is hard. I tried pretty much every heuristic I could think of (amount of high-frequency noise, color palette, size of uni-colored areas, amount of uni-colored areas, connectivity graphs etc.), but ultimately all of them failed. I did a bit of research on the internet and found quite a few papers on the subject, though it seems that most of them failed to find a stable solution, and the few solutions that actually work (i.e. recognition rate >80%) are way too complex to implement as a small hobby project.
Also, even if you manage to implement such an algorithm, it will still only differentiate between cartoon-images and real life ones. Detecting which pictures are pony and which aren't is a whole different story.

So, right now I'm just trying to write something that, under the condition that a picture is pony, will try to recognize which ponies are in it. This works well for ponies with unique features, but if you take a look at Fluttershy and Applebloom, for example, it's very hard to differentiate the two solely based on color. They have the same body color and the color of Applebloom's ribbon is almost the same as Fluttershy's mane color. So yeah, I think finding a solution to these problems is again too complex for a hobby project.

>>14313
>I did a bit of research on the internet and found quite a few papers on the subject,
Links plz!

Everything worth doing is hard.

>>14330
http://www.cs.unc.edu/~leejswo/vision/project_final_report.pdf
http://www.uv.es/~tzveta/invwork.pdf
Here's hoping that the wordfilter doesn't eat the URLs

>>14313
>>14349

Thanks for the information, Thunabrain. I didn't expect it to be that bad, especially given your earlier success for distinguishing cartoon/non-cartoon images. I also realized half way though that most computer vision tasks need to use domain specific knowledge to get anywhere. Its too bad your recent results are negative.

Still, although you've told us what CV features seems to work and not work, I don't really know what you are doing with these features for recognition afterwards. Both the report and thesis you linked to seem to (mainly) use support vector machines (SVMs). Is this what you've also used?

I don't doubt your conclusion, especially since both you and Roobles think that pony identification is easier than pony recognition, but some more information would be helpful. It would also be interesting to know what has worked best for now (even if the results are poor) and what accuracy rate you are getting.

It is interesting that you mention this Fluttershy/Applebloom problem since I've also noticed this with the (very few) examples I've tried through Roobles' web interface.

Hello, sorry for the long delay in posting. My life has been pretty hectic lately, and it looks like it's going to be that way for the next month or two. I'll try to post when I can, but I can't make too many commitments. Stupid life... getting in the way of my pony agendas...

Cantorlot:
Sorry about the source going down. It should be back up now. I don't have a very stable hosting solution at the moment, so my domain (callstack.org) is pointing to a computer sitting in my apartment. I was out of town for a week, and had what must have been a power outage. So, server went down, and I had no means of getting my websites back online until I got home again.

On Collaboration:
I'm all for collaborating on this project. I'm not sure though what languages everypony is using so far on this though. I get the feeling that C, the language I chose, may not have been the popular choice. But the current algorithms are pretty dang simple, and it could easily be written in any language. I would be quite open to providing checkin credentials to pony-net for you guys, or migrating the source to an alternative (more stable) public source control system. My one reservation so far though is not knowing if pony-net is accurate/good enough to even be worth collaborating on yet, or if I'd be wasting your time.

Pony-Net Progress:
Progress has been slow for me, but I'm still moving along. Most of my checkins have been surrounding the PHP scripts for genetic optimization, and I've been trying to utilize the down time in growing config populations. (I've got two computers at my work crunching pony numbers non-stop.) I've got a few sequences from it (some of them really, really weird and outside the boundaries of what I ever thought would be min/max values) that I'll probably try plugging into my web demo when I get the time.

I started work on the concept of a noise ratio, but my first stab at it (I was really going out on a limb with it) was a complete failure. All it did was make pony-net yield essentially unpredictable and chaotic results. I'll try a more sane implementation soon.

Beyond that, I've been working on a second program that's re-using all of my main header files. Its purpose is to read in a bunch of images and spit out the colors they have in common. I'm going to try to use it to figure out good values for defining the specific primary/secondary colors in ponies, but I don't know how well it will work. And... it's very, very processor intensive.

Still not yet to the point of trying to recognize pony shapes/features. But from reading the posts here so far, it looks like it's gonna be one heck of a doozy.

I've rented a VPS - I can set up us a Subversion (and anything else we need) there.

As for the choice of language and other things like that... I'm hoping we could arrange a brainstorming session on, say, Tinychat. When are you guys on? I'm awake from approx. 08:00 to 24:00 GMT every day.

I chose C# because
* it and .NET probably have the best documentation in the world (and if that fails, there's Jon Skeet)
* it's supported by, imho, the best IDE in the world: Visual Studio
* I don't even have to recompile it for Linux or OS X thanks to Mono
* my goal is a desktop application like Picasa - .NET's Windows Forms fits nicely
* also, garbage collection and protection from dangling pointers - overrated but worth mentioning

That said, I have no problem with C or any other imperative language as long as it's used properly.

>>14727
Good to see you back here. Real life keeps getting in the way of my pony agenda too.

I knew I should have checked out your code earlier. Well, at least nothing bad happened to your server (and the files on it), which was the most worrisome part. I will probably fiddle with it in a few days.

I might also try some approaches if I ever get the time (which seems to be never) for detect shapes (!) but only in the "basic" pose.

I can't really guess the details of Thunabrain's noise ratio classifier either (but then again, I should probably read up on it a bit beforehand).

For your new method that finds common colours, are you considering each image as a count (histogram) of colours and taking dot products or something? (Also, how many images are we talking about here?)

As for programming languages, I tend to favour scripting languages more but I'm fine with other choices too. Although, given how much code I've contributed, you should probably just pick something you are comfortable with (though php as a choice did seem a bit odd). Also, we don't have to stick to one language since most of our programs will be for data processing and we can just make them dump intermediate files if needed.

>>14747
Thanks for setting things up. I'm not as interested in an application as the algorithms though. But I don't think I've had much trouble using mono in the past so C# should be fine as well. (Although I haven't read much in the language, but I don't think that should be a problem here.)

Progress Update:
I finished the initial tweaking of the Pony-Colors program, and wrote a bash script to parse/feed it the training files from the genetic optimization scripts. It's naive, very exclusive, highly processor intensive, and not the least bit forgiving. But these are the files it spit out after I gave it the training set of high saturation screen shots from before:

http://callstack.org/pony-net/colors/Apple_Jack.html
http://callstack.org/pony-net/colors/Fluttershy.html
http://callstack.org/pony-net/colors/Pinkie_Pie.html
http://callstack.org/pony-net/colors/Rainbow_Dash.html
http://callstack.org/pony-net/colors/Rarity.html
http://callstack.org/pony-net/colors/Twilight_Sparkle.html

It caught a lot of colors I wasn't expecting; most turned out to be from the gradients in their eyes, colors from their cutie marks, or colors used in outlines. Something seems off with Rainbow Dash, but a single image can throw off the results right now. I need to add in some forgiveness, or choose better training files for now. (Note: the colors are currently ordered by my strange way of calculating an absolute hue value (which isn't actually used anywhere inside of Pony-Net;) so their order doesn't exactly reflect their prominence in the characters.)

I realized early on that color-based recognition is seriously limited by how accurate/representative the colors are in the definition file, but this is my first attempt to address that issue.

Also. This process is giving me some ideas. This program aims to find pony colors by only being given a set of images with the names of ponies inside each image. But there's no reason why the colors it finds can't be sorted by their presence, and fed into genetic optimization scripts to derive Pony-Net pony definition files. The ultimate goal being to pass in a set of images, and have the system automatically figure out how to recognize the named ponies, while simultaneously optimizing itself to become better at finding them. Processing power aside, I don't see any limitations yet as to why that wouldn't be a possibility. Later though. Still so much to do.

@Anon:
I'm generally awake and at a computer between 15:00 - 5:00 GMT. I can't really use TinyChat while at work, but I *can* use IRC/AIM. So I could be available most days 15:00 - 5:00 on IRC/AIM, or 23:00 - 5:00 on TinyChat. My AIM name is MacropusVulpes if anypony wants to get in touch with me. I'd likewise be available to hop on any public IRC server; GeekShed and FreeNode are the ones that come to mind though.

To be completely honest, I'm a bit weary of re-writing Pony-Net in C#. At the moment I don't have access to a Windows machine outside of work, and I do not own a license for Visual Studio. I also want pony-net to be completely free (I have plans of licesnsing it under either LGPL (preferable) or GPL (if forced to)) and I'm very concerned about the patent limitations on C#/mono (http://en.wikipedia.org/wiki/Mono_(software)#Mono_and_Microsoft.27s_patents). I also have concerns over the quality of the experience of developing C# applications inside of Linux from a CLI. I'm not sure if there is an ncurses-based debugger for C#, which I would need. (I very rarely code within a GUI.)

So languages I was thinking about (preferring) as alternatives to C would be:
C++, LISP, NASM (in small doses, if necessary)

With secondary preferences for:
Perl, Python

Tertiary:
PHP, Javascript, C#, Java, Objective C, Pascal, Ada

However. This doesn't mean our goals are mutually exclusive. We can definitely still work together!

My focus is on creating an efficient and accurate library. However that library is used, (GUI/CLI, client/server, etc) doesn't matter. So the core library could be plugged into a C# winforms client application (like you're envisioning,) just as easily as it could be used in a website (like the demo I hacked together in ten minutes), or as part of a continuous training environment. I can definitely look into what it would take to compile it to a dll so you could make calls to it from .Net. I would be really supportive/encouraging of such use, and very happy to see pony-net opened up to other platforms.

It likewise goes without saying that I'm still very open to collaborating on the library itself (under whatever name) and working to either rewrite it or plug it into a separate, more mature system. I would prefer though that the library be as open, free, and platform independent as possible though. So I just can't see it being written in a Microsoft dominated/controlled technology. (I'm really sorry if I come across as unfairly prejudiced; I just come with a lot of baggage and strong opinions. I still deeply respect people's decisions to work in any language or platform they wish to, and I recognize the benefits of C# .Net to a lot of developers.)

@Cantorlot:
Pony-Colors right now is pretty dang simple. (It's a single C file, even.) I have it reading in images, iterating through the pixels, and building an array of the "unique" colors used within the image. I have "unique" currently defined as a color match with thresholds: hue:1, saturation:20, lightness:10, (in accordance with the color algorithms I wrote.) It keeps track of a global list of those unique colors and compares it to each future image it reads in, keeping only the colors present in both. So, by the end of it, the only thing left is the list of colors that exist in all of the images.

"(though php as a choice did seem a bit odd)"
Yeah, it is odd. Pony-Net itself is in C. Just the training scripts are in PHP. I have no good reason for it. I just write in PHP. I don't know why. It's a bad habit that I need to kick, like some sort of abusive relationship. But it's always there, ready and convenient, familiar even if yet detestable. I was originally going to have the training be a simple bash script, but decided I may want something a little bit more robust. That's usually how my PHP misadventures start.

I'm at the point now where I'll likely be re-writing the training in a different language. I was on the verge of turning the training into a server/client model, whereby I could query or input commands to an active/running population, but got burned by the fact that PHP doesn't have multi-threading support. So I'm left with the choice now of forgoing server/client, implementing my own hacked multi-threading into PHP, or coming up with a hack solution using files on disk. Dropping PHP in favor of python seems the most reasonable choice.

>>15172
That's really nice that you've been looking for accurate colours (and for multiple reason).

Pony-Colors isn't part of PonyMatch is it?

I've been fiddling with this flood fill idea, which seems to be giving me regions just fine. The basic idea is that I want to use information about the (thick) outlines by looking colours of regions and their neighbours. I've been using your colour data (pony-match.cfg) which is why I'm glad to see you are trying to improve on the colours.

But I don't know what functions to use to determine how likely its a pony. Even my colour "distance" function uses plain RGB value differences (rather than hue, saturation, lightness as you have). I'm also not sure how to reward/penalize based on region sizes. Basically, I have a bit of (processed) data, I know how each parameter should correlate with the result I want but I don't know what function to pick globally. Its possible to wrap some AI around this (e.g., neural net, genetic, etc), but I was hoping to get something half decent hard coded first.

Actually, colour based recognition might not be a bad idea. For one, there doesn't seem to be much existing work on it (surprisingly, most CV technique seem to involve turning the input image into a greyscale image). So that what we could come up is likely to be completely new (which I think is nice).

One thing I've noticed is that the flood fill "method" (if you can call it that) seem to think that every eye is Rarity. This might make identifying eyes a viable option. This could give us the eye region with (hopefully) high accuracy and then identify ponies by eye colour. (I think in the show there are very few region adjacent to a medium-sized black and white region.)

Ultimately, we can probably just combine everything we can think of into one presence/certainty function.

I wasn't aware of C# licensing issues (since I haven't coded in it myself).

For Pony-Colors, I don't understand how one image can throw it off then. Unless you are grouping colours by the first one they see (so, for example, the order in which images are received will matter). Could you instead count all colour over all images for a given pony (i.e., no difference in hue/saturation/lightness)? And then sort them and print the top few? Would the results be about the same?

The remark about php was more of a joke. I don't think a bash script would have been enough for what train.php is doing. I guess if it become really big, then you might want to do something else. But even then, I wouldn't feel forced to.

I've finally looked at the code for pony-match. I was surprised by a few things. But I wouldn't think too much about it if I were you, especially since you got something working and I haven't. So its really meant as a "in case you haven't though about this". Also, some of this could be me misreading.

1.You seem to be only using libmagickcore to extract pixel values and nothing else. It might have useful features that you could use or you might be able to use a smaller library (that you could easily bundle with if the license are compatible).

2.You really are scanning line by line. I guess ponies do tend to be parallel to the axis. But then I'm wondering if you could instead store some summarized information about the lines (rather than going through them for every pony). I'm also wondering if libmagick has something that could do this faster. Unless this is not the bottleneck, of course.

I guess what I'm doing is an alternative to this, but it is also much slower.

3.You are storing the information for the (2d) image in a 1d array. Is this to ensure contiguous memory location or something? I'm not that familiar with how C works at a low level.

4.I don't really know what the training script is supposed to do. pony.pop doesn't seem to be used anywhere else. (In particular, I don't understand what optimization problem it is trying to solve.)

Sorry for not responding in so long. It's currently exam period, so I didn't have much free time to spend on the internets :)

>>14426
>I didn't expect it to be that bad, especially given your earlier success for distinguishing cartoon/non-cartoon images
Turns out that if you use a test dataset small enough, most simple algorithms will appear to work well if you tune them a bit. Needless to say, they fail miserably once you apply them to the general case. Silly me.

>I don't really know what you are doing with these features for recognition afterwards.
My original plan was to find a 'killer feature' of cartoon images based on which an algorithm could easily discern them from RL images. I didn't really do much research then, I just tested what seemed intuitive (e.g. RL images are unlikely to have large uni-colored areas, are more likely to use a larger color palette etc.)

>It would also be interesting to know what has worked best for now (even if the results are poor) and what accuracy rate you are getting.
What seemed to work best was first recognizing areas of roughly the same color, and then counting the number of areas and their size, or rather their respective ratio to the image size. Still, this usually only works for fanart with a few ponies and a plain background. Screenshots of the show tend to have busy backgrounds that contribute a lot to the number of areas found.

>>15074
>I can't really guess the details of Thunabrain's noise ratio classifier either
It's very simple. Apply an ordinary edge filtering algorithm (e.g. Sobel or Canny) and average the differentials/edges for the whole image. Again, this only works in some of the cases. Images from the show tend to be classified as real images due to busy backgrounds/jpg artifacts.

>>15587
This is exactly something I tested, and I didn't have much luck with it. Problem being that, for example, a purple background with a bit of a gradient can have the same colors as Twilight, or that the color palette of Fluttershy and Applebloom are very close. I tried solving some of this by estimating the size of the areas in relation to each other and their proximity (with least squares and connectivity graphs), but it still did not work as well as expected. I'd be excited to see it actually work though, so good luck!


Ultimately, like this anon mentioned >>14426, I think Support Vector Machines are the way to go. As far as I understood, you choose a set of image features that might help differentiate between cartoon/not cartoon, and while on their own they aren't of that much use, together they yield a high recognition rate. Then you just measure these features for each training image, get a p-dimensional vector and feed that to the SVM, which will then hopefully come up with a good estimate on how to separate the two classes of images.
I did a quick check on the internet and found a neat library called LIBSVM which has been ported to pretty much every major language (including C#, Java, Python, LISP and CUDA). You might want to have a look at that.

As I mentioned, I don't have that much time right now to work on this, but I think you guys have this covered. I might want to try the SVM later on though (if you haven't already by then).

So, it looks like the main reason the method I was trying didn't work well is because OpenCV stores pixels in BGR order rather than RGB. I discovered this thanks to Roobles' nice html bgcolor trick.

I'm getting much better results now. Its still a bit too slow to run on a good sized dataset. And its evaluation function probably still needs to be improved. It managed to get three ponies on the pic but for the moment thinks Fluttershy is Applejack's hair and Applejack is Applejack's hair outline.

I'll just post the code now and reply to posts later. I cleaned it up a bit but bad habits are all over the place. Please do not learn to code from me. It uses the Python wrapper for OpenCV.

The first (printed) output number is the region # of the best region. The second output represents certainty (the lower the better).

from cv import *
from __builtin__ import min,max,sum
import numpy as np

inputfile = "introsmall.png"
image = imread(inputfile)

newcol = CV_RGB(127,127,127)

lo = 20
up = 20
connectivity = 4
flags = connectivity + (255 << 8) + CV_FLOODFILL_FIXED_RANGE

def fullff(image):
regions = np.zeros((image.shape[0]+2,image.shape[1]+2),dtype=int)
mask = CreateMat(image.shape[0] + 2, image.shape[1] + 2, CV_8U)
Zero(mask)

cvRegion = np.zeros((image.shape[0]+2,image.shape[1]+2),dtype=int)
filled = image.copy()

regioninfo=[]
regionnum=1
linenum=0
while 1:
start = (0,0)
fillres = FloodFill(filled,start,newcol,CV_RGB( lo, lo, lo ), CV_RGB( up, up, up ), flags, mask)
regions = np.asarray(mask)
regioninfo.append(fillres)
AddS(cvRegion,1,cvRegion,mask)
regionnum+=1
while 1:
if np.all(regions[linenum]): linenum+=1
else: break
if linenum==len(regions): break
if linenum==len(regions): break
start = ((regions[linenum]==0).nonzero()[0][0]-1,linenum-1)

regioninfo.append((0,(0,0,0),(0,0,0,0)))
regioninfo.reverse()
return regioninfo,cvRegion

from pickle import load,dump

prefix = inputfile.split(".")[0]

#regioninfo = load(open(prefix+"-regioninfo.pkl"))
#cvRegion = load(open(prefix+"-cvRegion.pkl"))

regioninfo,cvRegion = fullff(image)
dump(regioninfo,open(prefix+"-regioninfo.pkl","w"))
dump(cvRegion,open(prefix+"-cvRegion.pkl","w"))

cvr = cvRegion[1:-1,1:-1]
print "Loaded"

def mcount(matrix,flatten=True):
count={}
if flatten: flat=matrix.flat
else: flat=matrix
for x in flat:
count[x] = count.get(x,0) + 1
return count

bigregions = [(i,inf[0]) for i,inf in enumerate(regioninfo) if inf[0]>500]

def getcol(cvr,rnum):
collist=map(tuple,image[cvr==rnum])
colcount=mcount(collist,False)
colsort=sorted(colcount.items(),key=lambda x:x[1])
return colsort

def bigregcol(cvr,bigreg):
brc=[]
for rnum,rsize in bigreg:
col,numpix=getcol(cvr,rnum)[-1]
brc.append((rnum,col,numpix/rsize))
return brc

def cdist(c1,c2):
return sum((c1[i]-c2[i])**2 for i in xrange(len(c1)))

def colconv(col):
return (int(col[5:7],16),int(col[3:5],16),int(col[1:3],16))

brc=bigregcol(cvr,bigregions)

matchdata = ({'SecondaryColors': ('#33346C', '#6D288C', '#EE4F90'), 'Outlines': ('#AD75C1', '#141F4D', '#010401'), 'Name': 'Twilight Sparkle', 'PrimaryColors': '#C8A9CF'}, {'SecondaryColors': ('#DC1E38', '#E35B29', '#FFFFC1', '#65CF4B', '#2D94C1', '#57087B'), 'Outlines': ('#6EAAD7', '#1F97CE', '#01030A'), 'Name': 'Rainbow Dash', 'PrimaryColors': '#ADEFFF'}, {'SecondaryColors': '#3E2F7F', 'Outlines': ('#CCD0D8', '#603778', '#83D2EF', '#010305'), 'Name': 'Rarity', 'PrimaryColors': '#F7FDFF'}, {'SecondaryColors': '#F867AA', 'Outlines': ('#F69CC8', '#DA4396', '#070003'), 'Name': 'Pinkie Pie', 'PrimaryColors': '#FBC5DF'}, {'SecondaryColors': ('#FAF3AD', '#CA9756'), 'Outlines': ('#E46E2B', '#ECD263', '#B2834D'), 'Name': 'Apple Jack', 'PrimaryColors': '#F9B763'}, {'SecondaryColors': '#FFBADA', 'Outlines': ('#EDDB6B', '#F592C5', '#030504'), 'Name': 'Fluttershy', 'PrimaryColors': '#FDF7A5'})

bigrnum=[x[0] for x in bigregions]

def addedge(g,v1,v2):
if v1 not in bigrnum or v2 not in bigrnum:
return
g[v1][v2]=g[v1].get(v2,0)+1
g[v2][v1]=g[v2].get(v1,0)+1

print "Building graph"

g={}
for rnum in bigrnum:
g[rnum]={}

def distadd(g,cvr,xdist=1,ydist=0):
maxx,maxy=cvr.shape
consecdiff = (cvr[:maxx-xdist,:maxy-ydist]!=cvr[xdist:,ydist:])
pairs = np.array((cvr[consecdiff],cvr[xdist:,ydist:][consecdiff])).transpose()
for a,b in pairs:
addedge(g,a,b)

distadd(g,cvr)
distadd(g,cvr,0,1)

distadd(g,cvr,0,2)
distadd(g,cvr,2,0)
distadd(g,cvr,1,1)

print "Finished building graph"

#lower is better
def valuefunction(primdist,nhdist):
big = 10**6
if len(nhdist)>1: return primdist+nhdist[0]+nhdist[1]
elif len(nhdist)>0: return primdist+nhdist[0]+big
else: return primdist+big*2

brcd = dict((b[0],(b[1],b[2])) for b in brc)

def getvalues(pc,outlines):
values = {}
for rnum in brcd:
rcol,rratio = brcd[rnum]
primdist = cdist(rcol,pc)/rratio
nhdist = []
for nh in g[rnum]:
rcol,rratio = brcd[nh]
nhdist.append(min(cdist(rcol,outline) for outline in outlines))
nhdist.sort()
values[rnum] = valuefunction(primdist,nhdist)
return values

for data in matchdata:
pc = colconv(data['PrimaryColors'])
outlines = map(colconv,data['Outlines'])
values = getvalues(pc,outlines)
print data['Name'],min(values.items(),key=lambda x:x[1])

#SaveImage("bestreg.png",(cvr==3030)*255)
#neigh=sum(cvr==x for x in g[3030])*255
#SaveImage("neigh.png", neigh)

@Cantorlot:
Pony-Colors is not part of Pony-Net (PonyMatch.) It does re-use color.h and image-get.h though, because I want it to always handle/analyze colors/images in the exact same way Pony-Net does.

It makes me happy to hear you're getting some use out of pony-match.cfg.

"For one, there doesn't seem to be much existing work on it (surprisingly, most CV technique seem to involve turning the input image into a greyscale image)."
That's what my initial research on the matter showed me. It's what caused me to say heck with it, and instead research color theory to write my own color analysis algorithms.

"This might make identifying eyes a viable option."
The difficulty I see in this is in the wide variety of shapes that eyes take throughout the gamut of expressions on the show. And a lot of pictures have ponies with closed eyes, whereby the eyelids take a shape mostly indistinguishable from other outlines, and don't offer color gradients for character recognition. (Or, the pony could even just be looking away.) It's something I want to take a look at in the future though, I'm just not sure how dependable it can be.

"Ultimately, we can probably just combine everything we can think of into one presence/certainty function."
Absolutely.

"For Pony-Colors, I don't understand how one image can throw it off then."
Pony-Colors currently finds the colors that are *common among all the images* (bitwise AND operation) presented to it. So, in the most extreme example: if you hand it an image with only a single color in it, it will discard the entire set of colors that the other images have in common.

"Could you instead count all colour over all images for a given pony (i.e., no difference in hue/saturation/lightness)? And then sort them and print the top few?"
This is possible. But would require giving it images where the main focus is the specific character you are trying to define. In group pictures, or images with large, diverse backgrounds, the colors of the character may likely be lost. There's a lot of background greens, browns, and blues in the show, which I believe would dominate and overpower the colors of the ponies themselves.

"I've finally looked at the code for pony-match."
*flutteryay*

"You seem to be only using libmagickcore to extract pixel values and nothing else."
That's absolutely correct. I chose libmagickcore because I know it has support for a very diverse set of image types. I'm fairly confident it can handle pretty much any image format thrown at it, reducing it to a set of pixels, and I like having the peace of mind that such an overhead (which is a daunting task in itself) is taken care of. And although I am not using it for other things right now, I may in the future. There are major performance hits as the image dimensions grow; so it may be beneficial in the future to use magickcore to resize the image data (which I'm sure it can do faster and more accurately than I can) before running it through any of the pony recognition algorithms. I also expect to be using it when I need to start addressing animated gifs (finding the "best" frame to analyze) and transparent images (it may be beneficial to change the background color that the transparent sections map to.) I figure that if Pony-Net makes it past alpha, and I'm still not using any other feature of libmagickcore, I'll likely look into replacing it.

"You really are scanning line by line... I'm wondering if you could instead store some summarized information... bottleneck..."
You've hit the nail on the head, and addressed a really important issue. You are correct, this is the bottleneck of the program right now, and it's terribly inefficient. I need to address it, but I don't feel ready to yet. The problem is that right now I don't yet know what part of the information I can safely summarize. The issue is that I plan to soon start scanning vertical lines, as well as horizontal, (possibly even diagonal), and combine the results. This will require gutting most of the actual scanning portion of pony-scan.c, and refactoring it to call and pass function pointers. So ummm... tldr: major changes will be happening soon, I don't fully know what they will be yet, but I plan to optimize once they're done. (I can elaborate more, if needed.)

"You are storing the information for the (2d) image in a 1d array. Is this to ensure contiguous memory location or something?"
Just personal preference. I had plans of looking at the data fowards, backwards, sideways, upside-down, whatever which way would give the best pony results. A two dimensional array felt unnecessarily biased towards reading it sequentially left-to-right-top-to-bottom. And I just find it easier to work with when it's a 1d array with a width and height.

"I don't really know what the training script is supposed to do."
The training script is meant to optimize Pony-Net's individual configuration settings. When you said you're, "not sure how to reward/penalize based on region sizes," that's the exact issue the script tries to address. To find the correct color thresholds, size/color expectations, etc to most accurately find the ponies listed in the training set. What gets used is the data that's printed to the screen or recorded in pony.log. So, for example, one of my current optimization scripts running has an entry of:

Fitness: 0.63 Sex: Female Genes: l:19 s:70 H:5 e:3 w:0.57 r:0.01 p:0.11 o:5

This means that with the configuration settings of a light/saturation/hue threshold of 19/71/5, an expected pony color count of 3, primary color bias of .57, primary color requirement of .01, a presence of 0.11, and a cuttoff of 5, pony-match had a 63% success rate at matching the ponies in the configuration file. ((+1 for pony matched that was there, -1 for pony matched that wasn't there) / total number of ponies in all images). This rating system is very susceptable to pony bias. I have plans for a pony balancing algorithm, to fight this though. I can elaborate more on that if interested.

For reference, current siblings of that sequence include:
Fitness: 0.39 Sex: Female Genes: l:19 s:70 H:5 e:3 w:0.57 r:0.01 p:0.01 o:5
Fitness: 0.57 Sex: Female Genes: l:19 s:127 H:5 e:3 w:0.57 r:0.01 p:0.11 o:6
Fitness: -0.02 Sex: Male Genes: l:19 s:70 H:5 e:3 w:0.57 r:0.00 p:0.11 o:5
(IE: These settings are damn fickle.)

Each generation, two things are logged: the most fit sequence yet obtained (which is appended to pony.log,) and the yet-to-be tested offspring (which are written to pony.pop.) Pony.pop is simply a serialization of the population datatype I use in the training scripts; it exists so I can interrupt the training, yet start back up right where it left off. Or, to take a population that has thrived on one training set, and introduce it to a different set.


@Thunabrain:
Real life unfortunately takes precedent to ponies. Good luck with your exams.

"Turns out that if you use a test dataset small enough, most simple algorithms will appear to work well if you tune them a bit. Needless to say, they fail miserably once you apply them to the general case. Silly me."
Story. Of. My. Life.

libsvm, you say? I'll certainly have to check that out. Thanks for brining it up.


Progress Update:
Pony-Colors now tracks and sorts by quantity. Added a color averaging algorithm to color.h, but initial attempts to use it in Pony-Colors have failed miserably. Going to have to build a linked-list based color map data type to track/store pony colors before I think I'm going to be able to use it effectively.

@Cantorlot:

Heh, you posted while I was writing my response (it takes me a while.) I have since looked at your code, and tried to get it running, but have come across a few obstacles. Right now the issue seems to be that whatever version of the python open-cv bindings I have installed doesn't contain the function "CreateMat". Still investigating that matter.

In the meantime, can you post the current results you're getting? I'm really interested in seeing it.

That's odd. But come to think of it, maybe I ended up using a (older) precompiled version of OpenCV in the end. Apparently, I'm using '$Rev: 3057 $'

>>> import cv
>>> cv.__version__
'$Rev: 3057 $'

Currently, there's only one line using CreateMat. You can actually replace the two lines

mask = CreateMat(image.shape[0] + 2, image.shape[1] + 2, CV_8U)
Zero(mask)

with

mask = np.zeros((image.shape[0] + 2,image.shape[1] + 2), dtype='uint8')

That was old code from a time when I couldn't figure out what format the numpy matrix needed to be to make it work in OpenCV. Actually, looks like the version I posted was also buggy (infinite loop!). Here are the new first few lines.

from cv import *
from __builtin__ import min,max,sum
import numpy as np

inputfile = "introsmall.png"
image = imread(inputfile)

newcol = CV_RGB(127,127,127)

lo = 20
up = 20
connectivity = 4
flags = connectivity + (255 << 8) + CV_FLOODFILL_FIXED_RANGE

def fullff(image):
regions = np.zeros((image.shape[0]+2,image.shape[1]+2),dtype=int)
mask = np.zeros((image.shape[0]+2,image.shape[1]+2),dtype='uint8')

cvRegion = np.zeros((image.shape[0]+2,image.shape[1]+2),dtype=int)
filled = image.copy()

regioninfo=[]
regionnum=1
linenum=0
start = (0,0)

while 1:
fillres = FloodFill(filled,start,newcol,CV_RGB( lo, lo, lo ), CV_RGB( up, up, up ), flags, mask)
regions = mask
regioninfo.append(fillres)
AddS(cvRegion,1,cvRegion,mask)
regionnum+=1
while 1:
if np.all(regions[linenum]): linenum+=1
else: break
if linenum==len(regions): break
if linenum==len(regions): break
start = ((regions[linenum]==0).nonzero()[0][0]-1,linenum-1)

regioninfo.append((0,(0,0,0),(0,0,0,0)))
regioninfo.reverse()
return regioninfo,cvRegion

from pickle import load,dump

And here's the output for the image I posted (introsmall.png).

Loaded
Building graph
Finished building graph
Twilight Sparkle (4297, 11679.004570237332)
Rainbow Dash (27737, 2627.6736020806243)
Rarity (18156, 3326.5756808106398)
Pinkie Pie (18755, 3369.3286679536677)
Apple Jack (19808, 7288.0)
Fluttershy (19225, 7225.5925925925931)

I'm another victim of real life infringing on my pony time. I'll have to respond to the other posts at a later time.

>>15985
Actually, this is probably a more useful image for you (in the same order as the output).

>>15632
>Turns out that if you use a test dataset small enough, most simple algorithms will appear to work well if you tune them a bit.
I'm also very afraid of overfitting. The "usual" way around this is to use a set (of images) for validation. These images are not looked at (by the algorithm or human depending on who's tuning parameters). Of course, if you look at these images afterwards, you will have to get a new validating set.

I haven't been doing this at all, having not even run the algorithm on a small set of images yet. So whatever I do is very prone to overfitting.

>Screenshots of the show tend to have busy backgrounds that contribute a lot to the number of areas found.
This kind of makes you appreciate the work that goes into this show.

>About edge detection.
Thanks for the details. I didn't know it was that simple. I might try (to add) that and fiddle with the parameters at some point.

>Problem being that, for example, a purple background with a bit of a gradient can have the same colors as Twilight, or that the color palette of Fluttershy and Applebloom are very close.
I don't really have any idea how I could ever distinguish Fluttershy from Applebloom (especially since there are sometimes shadows).

>About SVMs
I'm kind of glad you hadn't tried them yet which means there might still be hope (for some of the methods you've tried that failed).

>>15778
>Pony-Colors currently finds the colors that are *common among all the images* (bitwise AND operation)
Thanks for the clear explanation. I can definitely see how that could be a problem.

>This is possible. But would require giving it images where the main focus is the specific character you are trying to define.
Actually, I was hoping that the background colours vary while the pony colours remain more or less the same across images. So that if you give it enough images, we will find (hopefully correct) colours which stand out. But then again, from my own experiments, it looks like shadows could throw this off completely.

>That's absolutely correct. I chose libmagickcore because I know it has support for a very diverse set of image types.
Oh right. I forgot about that. (I've been using pngs only although I'm sure OpenCV can handle other things either.) That makes perfect sense.

>You've hit the nail on the head, and addressed a really important issue.
Well, I'm glad that the current bottleneck is something that we have control over and hopefully can be improved (depending on the set of information you will need).

>Just personal preference. I had plans of looking at the data forwards, backwards, sideways, upside-down, whatever which way would give the best pony results.
That's a good point, there's no reason to favour the xy-axes yet.

>Fitness: 0.63 Sex: Female Genes: l:19 s:70 H:5 e:3 w:0.57 r:0.01 p:0.11 o:5
Oh, I see. It's trying to find best (l,s,H,e,w,r,p,o) values. Since you are using AI methods, you can always use cross validation to avoid overtraining and get better estimates on performance (outside your sample).
http://en.wikipedia.org/wiki/Cross-validation_%28statistics%29

Just reserve a portion of the images to not train on and only check on those at the end. But now do the same many times. Split your images into, say 10 sets, and make each of them the validating set once (and the rest the training set).


On my side, I've been trying to add mane colours to what I was already doing, but was not very successful. I was trying to avoid putting hard thresholds on what is an outline region and what is not, but it seems many outline regions need to be crossed to go from head or body to mane. I could also use some mane (only) outline colours.

I have yet to read the full breadth of this thread aside from a few of the earlier posts and sadly I do not have the technical knowledge to fully understand what is being said.

But I know this. You sir, Roobles, are a gentlecolt and a scholar of the highest quality.

>>16661
Thankyee kindly, but I feel the praise is a bit misplaced. I'm just bumbling through what turned out to be a fairly recognized problem that several other bronies are trying to tackle as well. And as of yet, I haven't really provided anything tremendously useful that others can benefit from. If there's praise to be had at this stage, it should probably go to Cantorlot, Thunabrain, and the others that are more aggressively researching ideas and tools to help the cause.

@Cantorlot:
Thanks for providing the image and the CreateMat() workaround. I haven't made the time yet to get your source running, but I still plan to play around with it. The results are quite interesting though, and definitely show some promise so far.

I too am nervous of overtraining, but haven't done anything about it yet. Thanks for the wiki link; I'll look into cross validation. Right now I'd say the training scripts have a roadmap of: re-write in a language that supports multi-threading, implementation of pony balancing algorithm, and then cross validation/anti-overtraining measures.

Good luck on the mane/outline colors. One thing I feel worth pointing out is that Rarity's mane is particularly troublesome. I didn't realize it until I started work on these algorithms, but she's the only of the main cast that always has a gradient in her mane and tail. (Dang beauty-obsessed pony making things more complicated. Can't she spend less time on her appearance for the sake of (computer) science?)

Progress Update:
Had another trip this last weekend, so progress was hindered. But I started work on the color map data type, and I'm almost finished. It's based off a binary search tree of unsorted linked lists of pony colors. The linked lists are typically really short, whereas there can be tens of thousands of tree nodes. Already the base implementation's fake log(2,n) searches/insertions has shown incredible performance improvements over the previously hard coded n/2 searches and order n insertions. In the future, I can toss some auto-balancing algorithms into the mix, or convert it to a full blown b-tree implementation if I want to scrape some more performance. But for now at least, that's looking unnecessary.

Also, the added data that now gets stored stored, and the color averaging at completion, should definitely improve overall color accuracy. (Or so I hope.) I'll find out if that's true after I finish the color map intersection function.

Soon as the color map is finished though, I'll run some quick tests, publish my findings on pony colors, plug them into pony-match.cfg, and get back to working on Pony-Net proper.

Who's attending Mane6 livestream on Friday? ( http://www.mane6.com/2011/08/livestream-sneak-peek.html ) I believe it's a good opportunity to get together in TinyChat or IRC right after the event as everypony's gonna be online. What do you guys think?

Also, does anypony here have any experience with Source Forge or Google Code?

I was going to post something on the weekend but both power and phone line were out. The wind wasn't even that strong.

>>16738

>One thing I feel worth pointing out is that Rarity's mane is particularly troublesome.
I hadn't noticed this until you mentioned it! Thanks for pointing it out. Fortunately, her "outer mane" is a single colour. From a small dataset, recognition of her mane seems fine though. Ironically, my (current) algorithm has more trouble with Rainbow Dash. (Although, I know what could be done about it and your algorithm works well on Rainbow Dash.)

>But I started work on the color map data type, and I'm almost finished. It's based off a binary search tree of unsorted linked lists of pony colors.
This sound quite interesting.

>Soon as the color map is finished though, I'll run some quick tests, publish my findings on pony colors, plug them into pony-match.cfg, and get back to working on Pony-Net proper.
I'm looking forward to seeing this BST in action.

>>17803
I will most likely see that after the fact. I might be able to make it to a chat session at some point, although usually, I just have bits and pieces of time to look at this. IRC is prefered since it requires no account creation.

I do not have any particular recommendations for Sourceforge or Google Code, having used neither (although I did browse a lot of projects).

@Findpony: I've downloaded the code you've sent us. Unfortunately, findpony.exe doesn't seem to output anything. I ran it with

mono Release/findpony.exe 255 0 0 15 aghast.png

My guess is that it should return some value (maybe max?). Unfortunately, trainer.exe crashes horribly for me. Another part seems to require a webcam. Maybe if you could explain a bit what each part is suppose to do and what kind of input they can take. As I've mentioned before, I'm mostly interested in the algorithm, but was unable to find anything except the flood fill.


Here is the updated version of my code. It seems to be working well, but I haven't tested it on the appropriate scale yet. I forgot to mention earlier that the previous version was able to get all mane six on introsmall.png if I just reduce the penalty for not having two outline regions adjacent to the main region.

Basically, any total below 5000 indicates a pony is present and anything above 10000 indicates a copy is absent. Between those numbers is a bit ambiguous (usually it means no pony, I think).

I reverted to not looking for mane and body adjacencies. This, of course means that it thinks Fluttershy is present whenever Applejack('s mane) and Pinkie Pie are present. See attached picture which produces the output.

Input:screen/ajrarppts1.png
Loading
Loaded
Building graph
Finished building graph
Twilight 9439 1177.81604959
Rainbow Dash 2108 16473.7142857
Rarity 8247 1287.66304884
Pinkie Pie 4541 768.305591122
Applejack 3121 124.80104712
Fluttershy 6037 261.701492537
Rainbow Dash Mane 8247 30127.1983226
Rainbow Dash Mane 8247 29275.6975826
Rainbow Dash Mane 10330 42482.6666667
Rainbow Dash Mane 10946 37996.6030664
Rainbow Dash Mane 10946 40294.6848382
Rainbow Dash Mane 10082 36096.1507538
Rainbow Dash Mane 10082 47387.3969849
Rainbow Dash Mane 4841 24396.1621622
Pinkie Pie Mane 5338 923.61229611
Pinkie Pie Mane 5338 2035.75198662
Fluttershy Mane 4541 482.66581306
Fluttershy Mane 4541 989.769953052
Fluttershy Mane 4541 1144.95603927
Rarity Mane 8446 2398.62809917
Rarity Mane 8446 2158.0661157
Rarity Mane 8446 2056.7768595
Rarity Mane 7007 1678.44210526
Rarity Mane 7007 2735.78947368
Rarity Mane 7007 2256.29473684
Rarity Mane 8613 1720.15384615
Applejack Mane 4989 52.329915292
Applejack Mane 4989 28.9754792688
Twilight Mane 8961 2757.10300055
Twilight Mane 9988 649.877770316
Twilight Mane 9278 2080.45558087
Twilight Mane 7700 4058.44008795
Rainbow Dash Total 40869.8764479
Pinkie Pie Total 1691.91788723
Fluttershy Total 744.367305597
Rarity Total 2966.1051541
Applejack Total 153.776526389
Twilight Total 1827.69381991

It does seems to find correct regions for the other ponies present. Here's the source.

Oops. This message is too long. I will put the source in the next one.

Here's another regions image.
[code]
from cv import *
from __builtin__ import min,max,sum
from numpy import zeros
from collections import deque
from pickle import load,dump
import gzip
import sys
import os

loadgraph = True
loadregions = True

inputfile = "introsmall.png"

if len(sys.argv)>1:
inputfile = sys.argv[1]

print "Input:"+inputfile
image = imread(inputfile)

def fullff(image):
mask = np.zeros((image.shape[0]+2,image.shape[1]+2),dtype='uint8')

cvRegion = np.zeros((image.shape[0]+2,image.shape[1]+2),dtype=int)
filled = image.copy()

regioninfo=[]
regionnum=1
linenum=0
start = (0,0)

while 1:
fillres = FloodFill(filled,start,newcol,CV_RGB( lo, lo, lo ), CV_RGB( up, up, up ), flags, mask)
regioninfo.append(fillres)
AddS(cvRegion,1,cvRegion,mask)
regionnum+=1
while 1:
if np.all(mask[linenum]): linenum+=1
else: break
if linenum==len(mask): break
if linenum==len(mask): break
start = ((mask[linenum]==0).nonzero()[0][0]-1,linenum-1)

regioninfo.append((0,(0,0,0),(0,0,0,0)))
regioninfo.reverse()
return regioninfo,cvRegion

prefix = inputfile.split(".")[0]

if not os.access(prefix+"-regioninfo.pkl.gz",os.R_OK): loadregions = False
if loadregions:
print "Loading"
regioninfo = load(gzip.open(prefix+"-regioninfo.pkl.gz","rb"))
cvRegion = load(gzip.open(prefix+"-cvRegion.pkl.gz","rb"))
print "Loaded"
else:
regioninfo,cvRegion = fullff(image)
print "Saving"
dump(regioninfo,gzip.open(prefix+"-regioninfo.pkl.gz","wb"))
dump(cvRegion,gzip.open(prefix+"-cvRegion.pkl.gz","wb"))
print "Saved"

cvr = cvRegion[1:-1,1:-1]

def mcount(matrix,flatten=True):
count={}
if flatten: flat=matrix.flat
else: flat=matrix
for x in flat:
count[x] = count.get(x,0) + 1
return count

totalsize = image.size/3
bigrcutoff = int(0.0006*totalsize)
medrcutoff = int(0.0001*totalsize)

bigregions = [(i,inf[0]) for i,inf in enumerate(regioninfo) if inf[0]>bigrcutoff]
medregions = [(i,inf[0]) for i,inf in enumerate(regioninfo) if medrcutoff<inf[0]<=bigrcutoff]

def getcol(cvr,rnum):
collist=map(tuple,image[cvr==rnum])
colcount=mcount(collist,False)
colsort=sorted(colcount.items(),key=lambda x:x[1])
return colsort

def bigregcol(cvr,bigreg):
brc=[]
brcd={}
for rnum,rsize in bigreg:
cols = getcol(cvr,rnum)
col,numpix=cols[-1]
#dangerous for gradients
totalpix = 0
for x in reversed(cols):
if cdist(x[0],col)<200: totalpix += x[1]
else: break
brc.append((rnum,col,totalpix/rsize))
brcd[rnum]=(col,totalpix/rsize)
return brc,brcd

def cdist(c1,c2):
return sum((max(c1[i],c2[i])-min(c1[i],c2[i]))**2 for i in xrange(len(c1)))

def colconv(col):
return (int(col[5:7],16),int(col[3:5],16),int(col[1:3],16))

brc,brcd=bigregcol(cvr,bigregions+medregions)

matchdata = ({'SecondaryColors': ('#33346C', '#6D288C', '#EE4F90'), 'Outlines': ('#AD75C1', '#141F4D', '#010401'), 'Name': 'Twilight', 'PrimaryColors': '#C8A9CF'},
{'SecondaryColors': ('#DC1E38', '#E35B29', '#FFFFC1', '#65CF4B', '#2D94C1', '#57087B'), 'Outlines': ('#6EAAD7', '#1F97CE', '#01030A'), 'Name': 'Rainbow Dash', 'PrimaryColors': '#ADEFFF'},
{'SecondaryColors': ('#3E2F7F',), 'Outlines': ('#CCD0D8', '#603778', '#83D2EF'), 'Name': 'Rarity', 'PrimaryColors': '#F7FDFF'},
{'SecondaryColors': ('#F867AA',), 'Outlines': ('#F69CC8', '#DA4396'), 'Name': 'Pinkie Pie', 'PrimaryColors': '#FBC5DF'},
{'SecondaryColors': ('#FAF3AD', '#CA9756'), 'Outlines': ('#E46E2B', '#ECD263', '#B2834D'), 'Name': 'Applejack', 'PrimaryColors': '#F9B763'},
{'SecondaryColors': ('#FFBADA',), 'Outlines': ('#EDDB6B', '#F592C5'), 'Name': 'Fluttershy', 'PrimaryColors': '#FDF7A5'})

manes = {'Rarity':[(82,60,145),(84,60,147),(82,61,148),(85,60,125),(108,50,134)],
'Fluttershy':[(246,182,208),(233,166,202),(230,165,201)],
'Pinkie Pie':[(246,66,140),(235,92,155)],
'Applejack':[(253,246,176),(249,247,175)],
'Twilight':[(230,70,140),(40,55,115),(100,45,137),(59,75,135)],
'Rainbow Dash':[(254,247,170),(251,248,174),(242,112,50),(217,89,41),(234,66,65),(98,187,81),(29,152,214),(103,48,137)]}
manes['Rarity'].extend([(104,50,130),(93,30,120)])

manesoutline = {'Rarity':['#603778'],
'Fluttershy':['#F592C5'],
'Twilight':['#141F4D'],
'Applejack':['#ECD263'],
'Pinkie Pie':['#DA4396'],
'Rainbow Dash':['#1F97CE']}

matchd = dict((data["Name"],data) for data in matchdata)

def saver(rnum):
SaveImage("bestreg.png",(cvr==rnum)*255)
neigh=sum(cvr==x for x in g[rnum])*255
SaveImage("neigh.png", neigh)

bigrnum=[x[0] for x in bigregions]
bigrnum+=[x[0] for x in medregions]

def addedge(g,v1,v2):
if v1 not in bigrnum or v2 not in bigrnum:
return
g[v1][v2]=g[v1].get(v2,0)+1
g[v2][v1]=g[v2].get(v1,0)+1

def distadd(g,cvr,xdist=1,ydist=0):
maxx,maxy=cvr.shape
consecdiff = (cvr[:maxx-xdist,:maxy-ydist]!=cvr[xdist:,ydist:]).nonzero()
for i in xrange(len(consecdiff[0])):
x,y=consecdiff[0][i],consecdiff[1][i]
addedge(g,cvr[x][y],cvr[x+xdist][y+ydist])

def distadd(g,cvr,xdist=1,ydist=0):
maxx,maxy=cvr.shape
consecdiff = (cvr[:maxx-xdist,:maxy-ydist]!=cvr[xdist:,ydist:])
pairs = np.array((cvr[consecdiff],cvr[xdist:,ydist:][consecdiff])).transpose()
for a,b in pairs:
addedge(g,a,b)

def distaddall(g,cvr):
distadd(g,cvr)
distadd(g,cvr,0,1)

distadd(g,cvr,0,2)
distadd(g,cvr,2,0)
distadd(g,cvr,1,1)

print "Building graph"

if not os.access(prefix+"-g.pkl.gz",os.R_OK): loadgraph = False

if loadgraph:
g=load(gzip.open(prefix+"-g.pkl.gz","rb"))
else:
g={}
for rnum in bigrnum:
g[rnum]={}
distaddall(g,cvr)
dump(g,gzip.open(prefix+"-g.pkl.gz","wb"))

print "Finished building graph"

#lower is better
def valuefunction(primdist,nhdist):
big = 10**6
small = 500
if len(nhdist)>1: return primdist+nhdist[0]+nhdist[1]
elif len(nhdist)>0: return primdist+nhdist[0]*2+small
else: return primdist+big*2

def getvalues(pc,outlines,sc,bigreg=None,primonly=False):
if bigreg is None: bigreg=bigregions
values = {}
for rnum,rsize in bigreg:
rcol,rratio = brcd[rnum]
primdist = cdist(rcol,pc)/rratio
nhdist = []
for nh in g[rnum]:
rcol,rratio = brcd[nh]
nhsize = regioninfo[nh][0]
penalty = max(1,nhsize/rsize)
nhval = penalty*min(cdist(rcol,outline) for outline in outlines)
nhdist.append(nhval)
#outline intersection is greater than outline size
if g[rnum][nh]>regioninfo[nh][0]:
nhdist.append(nhval)
nhdist.sort()
values[rnum] = valuefunction(primdist,nhdist)
if primonly: values[rnum]=primdist
return values

ponyval = {}
for i,data in enumerate(matchdata):
pc = colconv(data['PrimaryColors'])
outlines = map(colconv,data['Outlines'])
sc = map(colconv,data['SecondaryColors'])
values = getvalues(pc,outlines,sc)
bestreg,bestval = min(values.items(),key=lambda x:x[1])
SaveImage("p"+str(i)+"-"+data['Name']+".png",(cvr==bestreg)*255)
print data['Name'].ljust(20),bestreg,bestval
ponyval[data['Name']] = bestval

bestmane = {}
for pname,pmanes in manes.items():
bestmane[pname] = "" #infinity
for mane in pmanes:
pc = tuple(reversed(mane))
outlines = map(colconv,manesoutline[pname])
values = getvalues(pc,outlines,sc,bigregions,False)
bestreg,bestval = min(values.items(),key=lambda x:x[1])
print (pname+" Mane").ljust(20),bestreg,bestval
bestmane[pname] = min(bestval,bestmane[pname])

for pname in manes:
print (pname+" Total").ljust(20),ponyval[pname]+bestmane[pname]

Forgot closing code.

from cv import *
from __builtin__ import min,max,sum
from numpy import zeros
from collections import deque
from pickle import load,dump
import gzip
import sys
import os

loadgraph = True
loadregions = True

inputfile = "introsmall.png"

if len(sys.argv)>1:
inputfile = sys.argv[1]

print "Input:"+inputfile
image = imread(inputfile)

def fullff(image):
mask = np.zeros((image.shape[0]+2,image.shape[1]+2),dtype='uint8')

cvRegion = np.zeros((image.shape[0]+2,image.shape[1]+2),dtype=int)
filled = image.copy()

regioninfo=[]
regionnum=1
linenum=0
start = (0,0)

while 1:
fillres = FloodFill(filled,start,newcol,CV_RGB( lo, lo, lo ), CV_RGB( up, up, up ), flags, mask)
regioninfo.append(fillres)
AddS(cvRegion,1,cvRegion,mask)
regionnum+=1
while 1:
if np.all(mask[linenum]): linenum+=1
else: break
if linenum==len(mask): break
if linenum==len(mask): break
start = ((mask[linenum]==0).nonzero()[0][0]-1,linenum-1)

regioninfo.append((0,(0,0,0),(0,0,0,0)))
regioninfo.reverse()
return regioninfo,cvRegion

prefix = inputfile.split(".")[0]

if not os.access(prefix+"-regioninfo.pkl.gz",os.R_OK): loadregions = False
if loadregions:
print "Loading"
regioninfo = load(gzip.open(prefix+"-regioninfo.pkl.gz","rb"))
cvRegion = load(gzip.open(prefix+"-cvRegion.pkl.gz","rb"))
print "Loaded"
else:
regioninfo,cvRegion = fullff(image)
print "Saving"
dump(regioninfo,gzip.open(prefix+"-regioninfo.pkl.gz","wb"))
dump(cvRegion,gzip.open(prefix+"-cvRegion.pkl.gz","wb"))
print "Saved"

cvr = cvRegion[1:-1,1:-1]

def mcount(matrix,flatten=True):
count={}
if flatten: flat=matrix.flat
else: flat=matrix
for x in flat:
count[x] = count.get(x,0) + 1
return count

totalsize = image.size/3
bigrcutoff = int(0.0006*totalsize)
medrcutoff = int(0.0001*totalsize)

bigregions = [(i,inf[0]) for i,inf in enumerate(regioninfo) if inf[0]>bigrcutoff]
medregions = [(i,inf[0]) for i,inf in enumerate(regioninfo) if medrcutoff<inf[0]<=bigrcutoff]

def getcol(cvr,rnum):
collist=map(tuple,image[cvr==rnum])
colcount=mcount(collist,False)
colsort=sorted(colcount.items(),key=lambda x:x[1])
return colsort

def bigregcol(cvr,bigreg):
brc=[]
brcd={}
for rnum,rsize in bigreg:
cols = getcol(cvr,rnum)
col,numpix=cols[-1]
#dangerous for gradients
totalpix = 0
for x in reversed(cols):
if cdist(x[0],col)<200: totalpix += x[1]
else: break
brc.append((rnum,col,totalpix/rsize))
brcd[rnum]=(col,totalpix/rsize)
return brc,brcd

def cdist(c1,c2):
return sum((max(c1[i],c2[i])-min(c1[i],c2[i]))**2 for i in xrange(len(c1)))

def colconv(col):
return (int(col[5:7],16),int(col[3:5],16),int(col[1:3],16))

brc,brcd=bigregcol(cvr,bigregions+medregions)

matchdata = ({'SecondaryColors': ('#33346C', '#6D288C', '#EE4F90'), 'Outlines': ('#AD75C1', '#141F4D', '#010401'), 'Name': 'Twilight', 'PrimaryColors': '#C8A9CF'},
{'SecondaryColors': ('#DC1E38', '#E35B29', '#FFFFC1', '#65CF4B', '#2D94C1', '#57087B'), 'Outlines': ('#6EAAD7', '#1F97CE', '#01030A'), 'Name': 'Rainbow Dash', 'PrimaryColors': '#ADEFFF'},
{'SecondaryColors': ('#3E2F7F',), 'Outlines': ('#CCD0D8', '#603778', '#83D2EF'), 'Name': 'Rarity', 'PrimaryColors': '#F7FDFF'},
{'SecondaryColors': ('#F867AA',), 'Outlines': ('#F69CC8', '#DA4396'), 'Name': 'Pinkie Pie', 'PrimaryColors': '#FBC5DF'},
{'SecondaryColors': ('#FAF3AD', '#CA9756'), 'Outlines': ('#E46E2B', '#ECD263', '#B2834D'), 'Name': 'Applejack', 'PrimaryColors': '#F9B763'},
{'SecondaryColors': ('#FFBADA',), 'Outlines': ('#EDDB6B', '#F592C5'), 'Name': 'Fluttershy', 'PrimaryColors': '#FDF7A5'})

manes = {'Rarity':[(82,60,145),(84,60,147),(82,61,148),(85,60,125),(108,50,134)],
'Fluttershy':[(246,182,208),(233,166,202),(230,165,201)],
'Pinkie Pie':[(246,66,140),(235,92,155)],
'Applejack':[(253,246,176),(249,247,175)],
'Twilight':[(230,70,140),(40,55,115),(100,45,137),(59,75,135)],
'Rainbow Dash':[(254,247,170),(251,248,174),(242,112,50),(217,89,41),(234,66,65),(98,187,81),(29,152,214),(103,48,137)]}
manes['Rarity'].extend([(104,50,130),(93,30,120)])

manesoutline = {'Rarity':['#603778'],
'Fluttershy':['#F592C5'],
'Twilight':['#141F4D'],
'Applejack':['#ECD263'],
'Pinkie Pie':['#DA4396'],
'Rainbow Dash':['#1F97CE']}

matchd = dict((data["Name"],data) for data in matchdata)

def saver(rnum):
SaveImage("bestreg.png",(cvr==rnum)*255)
neigh=sum(cvr==x for x in g[rnum])*255
SaveImage("neigh.png", neigh)

bigrnum=[x[0] for x in bigregions]
bigrnum+=[x[0] for x in medregions]

def addedge(g,v1,v2):
if v1 not in bigrnum or v2 not in bigrnum:
return
g[v1][v2]=g[v1].get(v2,0)+1
g[v2][v1]=g[v2].get(v1,0)+1

def distadd(g,cvr,xdist=1,ydist=0):
maxx,maxy=cvr.shape
consecdiff = (cvr[:maxx-xdist,:maxy-ydist]!=cvr[xdist:,ydist:]).nonzero()
for i in xrange(len(consecdiff[0])):
x,y=consecdiff[0][i],consecdiff[1][i]
addedge(g,cvr[x][y],cvr[x+xdist][y+ydist])

def distadd(g,cvr,xdist=1,ydist=0):
maxx,maxy=cvr.shape
consecdiff = (cvr[:maxx-xdist,:maxy-ydist]!=cvr[xdist:,ydist:])
pairs = np.array((cvr[consecdiff],cvr[xdist:,ydist:][consecdiff])).transpose()
for a,b in pairs:
addedge(g,a,b)

def distaddall(g,cvr):
distadd(g,cvr)
distadd(g,cvr,0,1)

distadd(g,cvr,0,2)
distadd(g,cvr,2,0)
distadd(g,cvr,1,1)

print "Building graph"

if not os.access(prefix+"-g.pkl.gz",os.R_OK): loadgraph = False

if loadgraph:
g=load(gzip.open(prefix+"-g.pkl.gz","rb"))
else:
g={}
for rnum in bigrnum:
g[rnum]={}
distaddall(g,cvr)
dump(g,gzip.open(prefix+"-g.pkl.gz","wb"))

print "Finished building graph"

#lower is better
def valuefunction(primdist,nhdist):
big = 10**6
small = 500
if len(nhdist)>1: return primdist+nhdist[0]+nhdist[1]
elif len(nhdist)>0: return primdist+nhdist[0]*2+small
else: return primdist+big*2

def getvalues(pc,outlines,sc,bigreg=None,primonly=False):
if bigreg is None: bigreg=bigregions
values = {}
for rnum,rsize in bigreg:
rcol,rratio = brcd[rnum]
primdist = cdist(rcol,pc)/rratio
nhdist = []
for nh in g[rnum]:
rcol,rratio = brcd[nh]
nhsize = regioninfo[nh][0]
penalty = max(1,nhsize/rsize)
nhval = penalty*min(cdist(rcol,outline) for outline in outlines)
nhdist.append(nhval)
#outline intersection is greater than outline size
if g[rnum][nh]>regioninfo[nh][0]:
nhdist.append(nhval)
nhdist.sort()
values[rnum] = valuefunction(primdist,nhdist)
if primonly: values[rnum]=primdist
return values

ponyval = {}
for i,data in enumerate(matchdata):
pc = colconv(data['PrimaryColors'])
outlines = map(colconv,data['Outlines'])
sc = map(colconv,data['SecondaryColors'])
values = getvalues(pc,outlines,sc)
bestreg,bestval = min(values.items(),key=lambda x:x[1])
SaveImage("p"+str(i)+"-"+data['Name']+".png",(cvr==bestreg)*255)
print data['Name'].ljust(20),bestreg,bestval
ponyval[data['Name']] = bestval

bestmane = {}
for pname,pmanes in manes.items():
bestmane[pname] = "" #infinity
for mane in pmanes:
pc = tuple(reversed(mane))
outlines = map(colconv,manesoutline[pname])
values = getvalues(pc,outlines,sc,bigregions,False)
bestreg,bestval = min(values.items(),key=lambda x:x[1])
print (pname+" Mane").ljust(20),bestreg,bestval
bestmane[pname] = min(bestval,bestmane[pname])

for pname in manes:
print (pname+" Total").ljust(20),ponyval[pname]+bestmane[pname]

I made a survey of available open source hosting services and decided to go with Google Code. SourceForge reviews all applications manually, so fuck 'em.

https://code.google.com/p/ponynet/

I've given Roobles admin access already. Everypony else needs to email me or Roobles an email address that has a Google account tied to it (protip: it doesn't have to be Gmail).

Progress has been rather slow. I've got quite a bit going on right now, but things should clear up for me after the 26th. In the meantime though, I was able to do some more color analysis. I'm still in the process of analyzing it and compiling it into a new pony-match.cfg.

@Cantorlot:
I started thinking about your recommendation for pony-colors to be inclusive instead of exclusive. And... I figured I'd give it a shot. I made some small tweaks, put together a sizable training set, and let it run. Here's the output:

http://callstack.org/pony-net/colors/inclusive/Apple_Jack.html
http://callstack.org/pony-net/colors/inclusive/Fluttershy.html
http://callstack.org/pony-net/colors/inclusive/Pinkie_Pie.html
http://callstack.org/pony-net/colors/inclusive/Rainbow_Dash.html
http://callstack.org/pony-net/colors/inclusive/Rarity.html
http://callstack.org/pony-net/colors/inclusive/Twilight_Sparkle.html

After giving it a look-over, I think this is the better approach. Thanks a bunch for suggesting it. But it requires lots and lots of pictures. The training set currently has 868, and it still feels much too small.

Explanation of the color files:
Each column is the output of a color map, where each color map is created using a different hue/saturation/light threshold. On the far left is 0:0:0, meaning that no grouping/averaging is performed. Every color there is an exact representation of the amount each color appears in the images. As it moves to the right, the thresholds get more loose, resulting in progressively more grouping/averaging. Threshold ratios are based on the genetic optimization tests I've run so far, where 5:70:19 was found to be optimal.

Can this be used to trace down a picture to the creeator's homepage?
Just in case we find something interesting but we don't know who made it, etc...

>>19748
What you're describing is called a reverse image lookup. Pony-Net can't do such a thing, and likely never will (it's just not its purpose.) But luckily there's already tools that do it. The most notable is Tin Eye: http://www.tineye.com/

Thanks for the new colour analysis, Roobles. As you can probably tell, things also got busier on my side. I will likely follow this project for a while without actually contributing to it (until I get enough time again).

callstack.org is down today so I wasn't able to look at the new data. What kind of accuracy are you getting with the inclusive method?

After going through Stanford's ml-class and ai-class courses, I'm restarting this project with new skills and determination.

First things first: WEB DEVELOPER NEEDED
If you're proficient with Flash and ActionScript, HTML5 Canvas and JavaScript, or Java applets, I need your help!

The first task in this project is to gather training data for the system. For that I need a web based MSPaint-like application that would let people quickly mark ponies in images. Web based so there's no installation needed and I can get it linked from EqD and crowdsource the entire fandom.

You were all taking too long, so a few weeks ago, I went and just did it.

Arise, Sweetie Bot.

Sweetie Bot, arise.

http://octabooru.net/ponident

Now with functioning eyes!

Single-class only for now, even though multi-class would be setting a flag. I want to do some special stuff to get multi-class working better before making that public.

How about trying to unrasterize the image? If it can be converted to vector graphics one could try and match properties on the strokes too.

Things like "swirlyness in mane" and "number of eyelashes" would be great inputs.

I am sure that it would be easy to think up a long list of small features that could be identified from a vector version.

>>42679
Wow, nicely done. I assume that it determines who it is based of colour alone? Well, I gave it a few goes and it's was 100% accurate, even on the ones that I thought would be tricky. I'm impressed.

>>42679

I almost feel like not posting because it would hide this link.

http://octabooru.net/ponident

Nice work! Have I seen this before? I've certainly seen octabooru before. I played with it for a bit and it only got 3 images wrong. And one of them was Gummy.

And the "extras" were are really nice touch.

I wish it showed confidence though.

>>42681

That sounds like a pretty good idea. To my knowledge, the closest that was tried is edge finding >>15632. I'm not sure what happens if the background is large (and swirly) compared to the pony. Unless it was just luck, I'd say Ponident works pretty well even when the pony is small.

how do i compile pony-net on windows?
i got minigw, bourne shell and php
when trying to compile i get these error messages:


C:Users******Desktopponymatch>sh make_projects.sh
make_projects.sh: pkg-config: command not found
pony-match.c:4:19: schwerwiegender Fehler: regex.h: No such file or directory
Kompilierung beendet.
image-get.c:1:31: schwerwiegender Fehler: magick/MagickCore.h: No such file or d
irectory
Kompilierung beendet.
pony-get.c:3:23: schwerwiegender Fehler: libconfig.h: No such file or directory
Kompilierung beendet.
pony-scan.c: In Funktion »GetScanPresence«:
pony-scan.c:71:42: Fehler: »__compar_fn_t« nicht deklariert (erste Benutzung in
dieser Funktion)
pony-scan.c:71:42: Anmerkung: jeder nicht deklarierte Bezeichner wird nur einmal
für jede Funktion, in der er vorkommt, gemeldet
pony-scan.c:71:57: Fehler: expected »)« before »ComparePresence«
make_projects.sh: pkg-config: command not found
image-get.c:1:31: schwerwiegender Fehler: magick/MagickCore.h: No such file or d
irectory
Kompilierung beendet.

and even though i downloaded imagick source and put it in the same directory it still tells me "No such file"

i'm also running it from the containing folder

could you help me getting this to compile or upload a windows binary?

So white-black ponies wouldn't work?

An apple.