A paper from researchers at Microsoft Research and Hebrew University details a new, spline-based algorithm for transforming pixel art, such as that from early sprite-based games, into scalable vector images.
Johannes Kopf and Dani Lischinski's "Depixelizing Pixel Art" [PDF] describes an upscaling technique that differs from popular methods used by companies like Adobe in ways that are particularly suited for the low-resolution sprites of classic games.
For example, the algorithm assumes that pixels in the initial image were placed deliberately by artists, and that lone pixels don't appear simply as artifacts of a digital imaging process.
The algorithm also evaluates pixel connections across the entire image, rather than focusing on a series of local areas, to determine whether pixels form a part of a long line that should be smoothed out or a corner that should be given a sharper edge.
The researchers have also implemented a series of heuristics to determine whether ambiguous pixel patterns, like a zig-zagging checkerboard, should be grouped together or separated into distinct lines.
Results for a variety of sample images presented in the paper improve drastically on existing algorithms, though the researchers admit their method fails for anti-aliased sprites like those found in Doom, and for situations where a more angular look is actually desired.
Scaling pixel art images to look natural on high-definition displays has been a problem for many modern updates to classic games, with most companies simply hiring artists to redraw those sprites in much greater detail, often at great cost.
The researchers hope future work will optimize their algorithm to the point where it can be used in real time on emulators, and even potentially form the basis for a method that generates additional frames to keep upscaled sprites from appearing jumpy.
The PDF Link is broken for me too, but if the picture in the article is an example of the upscaling, I would say, it doesn't look better, then what current emulators for 8 and 16 Bit machines are offering for years now.
The emulator coding community developed some very sophisticated filters, that are in use for ages.
It's been a while since I fired up the ol' emulators but I think you're comparing apples to oranges or maybe you misunderstand the intent behind what was done in the PDF.
The difference being that hq4x is "simply" upscaling the images (supposedly) which means that if you want to increase to full HD you have to keep upscaling it until you get the desired resolution. This technique convertis the images to vector graphics, allowing them to be upscaled to any resolution without further processing.
Good news all around! It'll make it easier to expose classic games to younger audiences who are turned off by highly pixellated images on high-def screens.
Even better, it could free up true, talented pixel artists from having to make a living through rehashing art assets that were conceived before they were, every time a publisher decides they want to release an updated retro-classics pack. Thus potentially pushing demand for original skilled pixel art into a new, more creative environment, as recently exemplified by Sword & Sworcery EP.
One small step toward an 8-bit renaissance! Intriguing, indeed! Hoo hoo!
If this or something similar were a part of Nintendo's new hardware or R&D, I'd feel that an "HD" Virtual Console could be viable. Renaissance, indeed!
"I'm not a fan of blurry vectory art, which is what this generates. It looses all the charm of the original."
Have to agree, especially when seeing this in motion with the Mario World video posted. I'd imagine it could work with some games or art styles, but with the Mario world examples (as well as many of the others), things started to look kind of blurry like they were made with blobs of colour. Just looking at the example on the last page, Mario's eyes don't look like anything more than two black dots. Ironically, there's a certain sharpness and clarity to the pixel art that's getting lost by trying to convert it to a higher resolution image with this technique and the original just ends up reading better visually if you ask me.
I agree with you. Pixel art is popular for a reason. I would love to draw some vectors and pixelate them (pixelate filters on Photoshop doesn't work the way I want).
This doesn't really work well. While it solves some of the problems of previous attempts at such algorithms, such as Super Eagle and hqnx (which had a tendency of turning circles into polygons), it still doesn't manage to make the end result look as good as the original pixel art.
Look at the algorithm being used in the context of a game:
http://www.youtube.com/watch?v=o2Fd-4NzB0w
It managed to make Super Mario World, what in my eyes is still a very appealing looking game, look like a cheap flash game.
I don't think it's actually possible to make an algorithm that does this well, at least not without involving computer vision AI in some way. The problem is is that making pixel art isn't a matter of simply scaling down high-resolution art. If that were true, computers could do it, which is still not the case. As such, creating high-resolution art out of pixel art isn't a matter of simply scaling it up and applying a few filters.
I agree with you, the depixelated result simply eliminated to much detail, the original graphics had. It seems impossible for an algorithm to determine, when a certain order of pixel should be treated as information that should be preserved and when it is just "noise", that can be smoothed out. It would be necessary to tell the algorithm how important every pixel is, and in this case, I think it is easier to redraw the graphics.
Additionally all these filters don't take into account, where the graphics were supposed to be displayed on a CRT-TV set and not a TFT HD monitor. Ian Bogost developed an interesting filter for the VCS emulator Stella, that took the intended look in consideration, but if you look at the different versions of this CRT-Filter, in the different versions of Stella, you can clearly see, how much interpretation is involved in such approach.
I don't know, the SMW demo looked good to me aside from the framerate. Perhaps artists could go over an upscaled version, add more pixel art, upscale some more, etc a few times for an ideal result. At that point, it might be easier to start from scratch though *shrug*.
That's like saying that high-framerate movies look like cheap TV soap operas...
It's undeniably an improvement, but people don't like it just because it reminds of something they don't like that just happened to have a better tech because it was invented later..
I think that SMW demo looks great, despite losing all the pixelated retro charm.
What's the use? 3d textures today are already in resolutions, that look photo-realistic, using lower resolutions and scale them up, would make them look awful.
If you want to work on the future of 3d textures, look at procedural textures.
But where would it help optimizing textures on games, that are meant to look cartoony?
I don't see any problem with current texture sizes in this kind of games, especially, when I take into account, that the above algorithm works best in high resolutions, that come on devices, that already have enough memory and processing power to handle such textures without any problem.
The emulator coding community developed some very sophisticated filters, that are in use for ages.
I don't know, what stands in the PDF, cause the link is broken.
It is just a matter of using them.
Even better, it could free up true, talented pixel artists from having to make a living through rehashing art assets that were conceived before they were, every time a publisher decides they want to release an updated retro-classics pack. Thus potentially pushing demand for original skilled pixel art into a new, more creative environment, as recently exemplified by Sword & Sworcery EP.
One small step toward an 8-bit renaissance! Intriguing, indeed! Hoo hoo!
more coffee now
http://games.slashdot.org/story/11/05/24/2355239/Upscaling-Retro-8-Bit-Pixel-Art
-To-Vector-Graphics
I'm not a fan of blurry vectory art, which is what this generates. It looses all the charm of the original.
Anyways, I'v never had a problem with SD content on my HD set, because I bought a plasma.
Have to agree, especially when seeing this in motion with the Mario World video posted. I'd imagine it could work with some games or art styles, but with the Mario world examples (as well as many of the others), things started to look kind of blurry like they were made with blobs of colour. Just looking at the example on the last page, Mario's eyes don't look like anything more than two black dots. Ironically, there's a certain sharpness and clarity to the pixel art that's getting lost by trying to convert it to a higher resolution image with this technique and the original just ends up reading better visually if you ask me.
Look at the algorithm being used in the context of a game:
http://www.youtube.com/watch?v=o2Fd-4NzB0w
It managed to make Super Mario World, what in my eyes is still a very appealing looking game, look like a cheap flash game.
I don't think it's actually possible to make an algorithm that does this well, at least not without involving computer vision AI in some way. The problem is is that making pixel art isn't a matter of simply scaling down high-resolution art. If that were true, computers could do it, which is still not the case. As such, creating high-resolution art out of pixel art isn't a matter of simply scaling it up and applying a few filters.
Additionally all these filters don't take into account, where the graphics were supposed to be displayed on a CRT-TV set and not a TFT HD monitor. Ian Bogost developed an interesting filter for the VCS emulator Stella, that took the intended look in consideration, but if you look at the different versions of this CRT-Filter, in the different versions of Stella, you can clearly see, how much interpretation is involved in such approach.
That's like saying that high-framerate movies look like cheap TV soap operas...
It's undeniably an improvement, but people don't like it just because it reminds of something they don't like that just happened to have a better tech because it was invented later..
I think that SMW demo looks great, despite losing all the pixelated retro charm.
What you say, is that high-framerate movies look better, because of the higher framerate, not because of what is displayed.
I think we're talking about different people saying different things.
I was talking about displaying the same content with 2 different techs, and why some people don't like the new version.
Why focus on emulators and old 2d games,
this tech needs to make its way into 3d texture rendering for future games.
If you want to work on the future of 3d textures, look at procedural textures.
Not everything is photo-realistic or meant to be.
I don't see any problem with current texture sizes in this kind of games, especially, when I take into account, that the above algorithm works best in high resolutions, that come on devices, that already have enough memory and processing power to handle such textures without any problem.
If you have no need for that, lucky you!