It is currently Sun Aug 25, 2019 3:20 pm




Post new topic Reply to topic  [ 18 posts ]  Go to page 1, 2  Next
New Vision 2.0 - Deus Ex, AI, and you 
Author Message
New Vision
New Vision
User avatar

Joined: Sat Nov 19, 2005 10:03 am
Posts: 2352
Post New Vision 2.0 - Deus Ex, AI, and you
Chances are some of you have heard of the hot new emulation trend involving AI upscalers. Rather than using a basic algorithm like bicubic sampling, this uses a trained AI model to estimate what detail is and how best to scale it up to a higher resolution. This is mostly intended for photos, but it's had some success with game textures as well.

You can see where this is going; over the past day I've been dicking around with these using the Deus Ex textures, and the results are pretty promising. The most common system used in 'the scene' is ESRGAN (since it's free), but I didn't think the results were very good for textures. Instead I've been using a commercial application, Topaz A.I. Gigapixel, and the results are pretty good.

Some comparison screenshots with Vanilla / New Vision 1.5 / New Vision 2.0 (i.e. AI Upscale):
https://imgsli.com/MjY4Mw
https://imgsli.com/MjY4NQ
https://imgsli.com/MjY4Ng
https://imgsli.com/MjY4Nw
https://imgsli.com/MjY4OA
https://imgsli.com/MjY4OQ
https://imgsli.com/MjY5MQ
https://imgsli.com/MjY5Mg
https://imgsli.com/MjY5Mw
https://imgsli.com/MjY5NA

What's so great about this?
    -It's ALMOST completely automatic, for most textures. Textures can be up-res'ed in batches and quickly previewed in game using the DX10 sideloading feature
    -It maintains the original art style and direction of Deus Ex, as there's no other artist's hand involved
    -It can be used on literally all game textures, including character models, which HDTP never had the manpower to touch
    -This is the big one: this can be made to work with any mod, using the same batch export / conversion process.

What are the limitations?
Well, it doesn't look as high fidelity as New Vision, obviously. There's limits to what the AI can do, especially as it's built on pattern recognition in photos. There are several factors that limit how successful the upscale is on textures, like the original resolution, how 'abstract' it is, etc.

I also haven't figured out a good technique for masked textures yet.

Enough talk - I want to try it out!

DOWNLOAD HERE

Alright, so to test this out you'll want to uninstall any other graphical mods. That includes New Vision 1.5 and HDTP.

Take the contents of the archive and put them in your DeusEx\Textures folder.

Launch the game with Kentie's Direct X 10 Renderer - this is important, as I'm currently using his sideloading technique to load the textures.

This leads to some jumpiness in game; this would disappear if I used the original method of DXT Merging; so it's a solvable problem.

Also thanks to Björn for helping me extract all the original textures using Hanfling's UCC script.


Sun Feb 17, 2019 12:43 am
Profile
MJ12

Joined: Thu Apr 02, 2009 7:55 pm
Posts: 464
Post Re: New Vision 2.0 - Deus Ex, AI, and you
First of all, it great seeing you working on NV! Welcome back!

Now, the thing about AI upscalers is that it makes everything clearer. It looks great in UNATCO and places that are supposed to look shiny and, well, clear - but it doesn't work well when things are supposed to look grimy and it indeed hurts the artistic vision. For instance, the first image (UNATCO) looks superior to everything before it, while the second image (Brooklyn tunnels) looks inferior.

I feel like just AI upscaling everything will make for a lesser definitive version. It could prove to be better if more post processing would be done afterwards - re-adding the grime, dust, and maybe noise. Heck, maybe it'll even be easier to just mix and match NV1.5's textures with the AI ones to have the most complete texture set. This is also extremely true for the mangled Chinese parts that NV1.5 fixed.

Just my $.02, though - you really know more about it than me. And once again, glad having you back!


Sun Feb 17, 2019 12:20 pm
Profile
MJ12
User avatar

Joined: Sat Jul 10, 2010 10:06 am
Posts: 312
Location: Finland
Post Re: New Vision 2.0 - Deus Ex, AI, and you
Made in China wrote:
Now, the thing about AI upscalers is that it makes everything clearer. It looks great in UNATCO and places that are supposed to look shiny and, well, clear - but it doesn't work well when things are supposed to look grimy and it indeed hurts the artistic vision. For instance, the first image (UNATCO) looks superior to everything before it, while the second image (Brooklyn tunnels) looks inferior.


I think it indeed would be hit-and-miss in certain areas of Deus Ex (however, UNATCO, Hong Kong, Area 51 and the character models look quite fantastic, though).

But since concepts of such fidelity don't exist in Deus Ex: Nihilum... well, damn. And it hasn't even been two months since I released the "final version". +o(

Thanks for the heads-up on this one, DaveW, I'll test NewVision2a.7z out as soon as I can!

_________________
DXN - Deus Ex: Nihilum


Sun Feb 17, 2019 5:52 pm
Profile WWW
New Vision
New Vision
User avatar

Joined: Sat Nov 19, 2005 10:03 am
Posts: 2352
Post Re: New Vision 2.0 - Deus Ex, AI, and you
I think it should be possible to mix and match; I personally wouldn't mix and match New Vision 1.5 and these textures, though, since they are vastly different styles and there is no way you could isolate a certain area of the game (e.g. have a grungy New York, but not UNATCO). Hong Kong is probably the only area in the game that is almost completely standalone in it's texture set.

You could, though, use the character textures with HDTP keeping the original character models. The AI upscaler works really well for most characters since it can determine detail like hair, faces, etc.

The way I picture releasing this, is that there's the main release and then subsequent 'mod packs' that will replace specific mod textures. I'm not sure how many of these I would do personally, but I would make sure the instructions are there for anyone to do them. The only difficulty is with masked textures which requires a bit of a manual touch with Photoshop.


Sun Feb 17, 2019 7:14 pm
Profile
MJ12

Joined: Thu Apr 02, 2009 7:55 pm
Posts: 464
Post Re: New Vision 2.0 - Deus Ex, AI, and you
I agree that it looks great on character textures. There are some edge cases on Tiffany Savage and Mechs - as they do have some unnatural elements to them (camouflage and mech parts) which might looks even more unnatural after the AI upscaling - which might prove beneficial to the latter, and detrimental to the former.

Do you think that going over and applying a layer of dust or grime using will be too much work? Because I feel like that intelligently adding those layers will be the best solution, if the same texture isn't reused in an inappropriate context.


Sun Feb 17, 2019 7:40 pm
Profile
New Vision
New Vision
User avatar

Joined: Sat Nov 19, 2005 10:03 am
Posts: 2352
Post Re: New Vision 2.0 - Deus Ex, AI, and you
Even with the mechs, the AI seems to pick up on the detail fairly well because the original textures are comprehensible. It only struggles when the original image is vague or really low resolution, which is the case for 50% of the environment textures. It doesn't help that Deus Ex has a really 'shiny' art style either.

Image

An extra grunge pass might be something someone else can do; to be honest I'm 90% done, I just need to package it up with the last few manually updated textures. But I'm going to make a tutorial so someone else can follow this up if they want to.

I'm updating the download link, it now includes all the masked textures, so characters, deco objects, and third person weapons are now complete.


Mon Feb 18, 2019 9:20 am
Profile
Off Topic Productions
Off Topic Productions
User avatar

Joined: Sat Apr 24, 2004 9:21 pm
Posts: 14217
Location: Hafnia
Post Re: New Vision 2.0 - Deus Ex, AI, and you
This is really cool, damn! Nice work :o

_________________
Jonas Wæver
Chief Poking Manager of TNM

You can buy my games on Steam if you like 'em.
Expeditions: Viking
Expeditions: Conquistador
Clandestine


Mon Feb 18, 2019 11:00 am
Profile WWW
MJ12
User avatar

Joined: Sat Jul 10, 2010 10:06 am
Posts: 312
Location: Finland
Post Re: New Vision 2.0 - Deus Ex, AI, and you
Agreed! Well, it's not always an automatic success, as with these Queen's Tower signs:

Attachment:
Shot0000.jpg
Shot0000.jpg [ 210.35 KiB | Viewed 4052 times ]


But overall, very very nice.

Is there a major difference in quality if the original texture is enlarged to, say, 1024x1024 or above instead of 512x512 like in the zip?

_________________
DXN - Deus Ex: Nihilum


Mon Feb 18, 2019 7:20 pm
Profile WWW
New Vision
New Vision
User avatar

Joined: Sat Nov 19, 2005 10:03 am
Posts: 2352
Post Re: New Vision 2.0 - Deus Ex, AI, and you
Pretty much anything with text scales quite poorly, due to how low-res and blurry the originals were. So there's a bunch of signs like the UNATCO nameplates that look a bit rubbish, and could probably do with a manual remake. I've already manually recreated some of the more basic text signs (e.g. ones that are text over a black background) but these would require a lot more work which I'm not feeling up for with my other projects taking priority.

The 4x upscale is actually a limit of Topaz AI which seems to cap at 6x resolution, but when I've forced it to upscale to 1024x1024, the results are actually worse. On some of the textures where it struggles, you get what looks like JPEG compression artifacting, which is a lot more obvious at a high resolution. It's actually better to leave it at 512 and then this detail kind of 'blurs out' as the game upscales the texture.


Mon Feb 18, 2019 7:35 pm
Profile
MJ12
User avatar

Joined: Sat Jul 10, 2010 10:06 am
Posts: 312
Location: Finland
Post Re: New Vision 2.0 - Deus Ex, AI, and you
Alright, thanks for the heads-up.

Would be really cool if the issue with enlarging masked textures could be solved, someone could possibly finish the UI rescaling projects (started by kocmo and the like) with the aid of this tool.

_________________
DXN - Deus Ex: Nihilum


Mon Feb 18, 2019 7:57 pm
Profile WWW
New Vision
New Vision
User avatar

Joined: Sat Nov 19, 2005 10:03 am
Posts: 2352
Post Re: New Vision 2.0 - Deus Ex, AI, and you
It's mostly a case of finding the right process; I've found that up-scaling the image with the usual pink mask colour and then using Photoshop's Select and Mask tool (with colour decontamination) is a fairly efficient way of doing it. It's not automatic, but it is quite fast.

I doubt this technique would work for the UI, since it's relying on pattern recognition with a model trained on photos. Stuff like that will still have to be done manually, though the D3D10 renderer side-loading would make it relatively simple to load those textures without messing around with the .u files.


Tue Feb 19, 2019 3:54 pm
Profile
NSF

Joined: Sat Nov 05, 2016 1:18 pm
Posts: 80
Post Re: New Vision 2.0 - Deus Ex, AI, and you
This is awesome, thanks so much! Great to see you modding again <3.

I've actually been looking into this sorta thing for a while for the community update https://trello.com/c/U1DaBwEd/46-experi ... resolution.

I've tried waifu2x and the results were OK, but wanted to experiment with other tools. It's great to see what Gigapixel looks like!

Certainly seems better then w2x!

Do you mind if I include this an an option for the Community Update? It's really great, and a lot more lightweight then NV 1.0!

I think I might get Gigapixel myself, looks like it might be the best / easiest option; would love to be able to help with this.

Also, posted about this to https://old.reddit.com/r/GameUpscale/ and r/Deusex, thans again for all the work you've done for the DX Community!


Thu Feb 21, 2019 12:09 am
Profile
New Vision
New Vision
User avatar

Joined: Sat Nov 19, 2005 10:03 am
Posts: 2352
Post Re: New Vision 2.0 - Deus Ex, AI, and you
Oh, neat - I didn't know there was a subreddit dedicated to this kind of thing. I stumbled upon Gigapixel months ago by accident and only recently saw people experimenting with ESRGAN for game textures.

The results definitely vary a lot based on the model; most models (at least, the example ESRGAN and Gigapixel) rely on photos so are great at extracting real world detail from textures, but when they're more abstract it struggles. Hopefully as more people build models trained on actual game textures (there seems to be an effort to do this) we'll start seeing better results. I'm going to upload a sort of mini-SDK with all the Deus Ex textures extracted so people can quickly run different models on the Deus Ex texture set and test it out.

I've got no problem with you including it in the community update, this version isn't final though, I might go back and create some more manual textures and also improve some of the masked ones.


Thu Feb 21, 2019 3:49 am
Profile
Illuminati
User avatar

Joined: Fri Mar 20, 2009 3:01 pm
Posts: 3937
Location: Terra, Sweden, Uppsala.
Post Re: New Vision 2.0 - Deus Ex, AI, and you
I actually think 1.5 looks better than ai upscale.

2.0 is closer to the original, but 1.5 has more detail and overall sells the scenes better imo.

_________________
"Delays are temporary; mediocrity is forever."
odio ergo sum


Sun Feb 24, 2019 1:53 pm
Profile
NSF
User avatar

Joined: Sun Nov 18, 2012 4:19 pm
Posts: 85
Location: Brussels
Post Re: New Vision 2.0 - Deus Ex, AI, and you
Holy Smoke, this is promising! My subjective opinion:

I agreed with Jaedar that this upcoming v.2.0 looks globally more faithful to original Deus Ex, I like it very much. Previously depending on areas, textures (especially floors) looked garbled sometime. This reminds me how ENB and SweetFX tools (nothing to do AI Upscalers, I guess) could often "shave off" textures and deprive them from their substance.

Also I agree that the effect on characters, especially their faces, is truly impressive: they look sharper and more vivid.

DaveW wrote:
...this version isn't final though, I might go back and create some more manual textures and also improve some of the masked ones.


Good catch. Considering your solid credentials with previous versions of New Vision, I'm confident that you'll make something worthy. I bet you wouldn't go out of your way to work on a new version of "New Vision" if this AI Upscaler wasn't so full of promise. So sincere Good Luck on your endeavours, friend.

Finally, DaveW, my dear! I’m so happy to hear from you again. For the record you're one of the reasons why I registered here in 2012: just to thank you for "New Vision". In other words:

"Great Modders are just like retired Rock Stars,

sometimes they do come back!" :mrgreen:


Defaultplayer001 wrote:
...I've tried waifu2x and the results were OK...


Well! Out of context that’s a rather funny statement. ^^

Thanks for the pointer though, this tool sounds super useful and I might just need it.

_________________
Pity the world if you fail Spirit Monk, for you are the last. (Jade Empire)


Mon Feb 25, 2019 2:06 pm
Profile
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 18 posts ]  Go to page 1, 2  Next


Who is online

Users browsing this forum: No registered users and 1 guest


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
Powered by phpBB © phpBB Group.
Designed by Vjacheslav Trushkin for Free Forum/DivisionCore.