Sign in to follow this  
Followers 0
chown

Common sub domains

8 posts in this topic

About a year ago I wrote a very basic C subdomain scanner (I'm sure just about everyone has written one of these), and now I'm updating it - implementing multithreading, load ballancing, and one or two other bits & pieces. So now I'm recompiling my list of common subdomains (see subdomains.txt), and if any of you could contribute a couple I've missed, that'd be great. :)

subdomains.txt

Oh, and here's original (old) source, so you guys can screw around with it as you see fit. It's a bit ugly, but hey. I hacked it together in about half an hour. (Don't forget to link ws2_32)

main.c.txt

Edited by chown
0

Share this post


Link to post
Share on other sites

Storing all of the subdomains in a linked list seems a bit useless as you are only going to cycle through once in an execution of the application. It would be faster and more efficient just to separate the subdomains by lines in the text file and read them right out of there like you currently are.

0

Share this post


Link to post
Share on other sites

Ah beautiful, thanks.

@livinded:

Yes, it'd be pointless to implement the idea in the old linear code, but in the new version I'm using linked lists so that I know the size of the list and can assign parts to different processes to allow parallel DNS resolution.

0

Share this post


Link to post
Share on other sites

If you are using threads to parallelize it and the threads don't each have their own file descriptor, it wouldn't matter. all you have to do is pass the file descriptor to the thread, lock the file while the thread reads it, unlock the file and then allow the next thread to read from it. It's not like as soon as you read some the descriptor is going to move the next byte back to the beginning. It will stay right after the last byte read until the descriptor is closed or you call rewind(), ungetc(), or some other function of that sort.

0

Share this post


Link to post
Share on other sites

That doesn't allow me to know the total number of entries in the list without digesting the file prior to the main loop.

At the moment, I load the file into a linked list to determine that it contains X items, then assign X/TOTAL_THREADS*n through to X/TOTAL_THREADS*n+X/TOTAL_THREADS to thread n.

Because I know the subdomain list contains X items, I can also (more easily than otherwise) display the progress in percentage.

0

Share this post


Link to post
Share on other sites

blah blah blah blah blah why are you even using C for this?

0

Share this post


Link to post
Share on other sites

Because I like C. It does what you tell it to do ^_^

Here's the full list so far. (Sorted and dupes removed)

subdomains.txt

Infact here's the script I wrote to do that:

(Javascript)

function removeDups(field) { 
if(field.value) try {
var tmp = '';
var foo = ('\n' + field.value.replace(/\n{2,}/g, '\n') +
'\n').split('\n').sort().join('\n') + '\n___';

while((foo = foo.replace(/\n([^\n]+)\n\1\n/g, '\n$1\n')) != tmp) {
tmp = foo;
}

field.value = foo.replace(/\n{2,}/, '').replace(/(.*)\n___$/, '$1');
} catch(e) {alert(e)}
}

Edited by chown
0

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!


Register a new account

Sign in

Already have an account? Sign in here.


Sign In Now
Sign in to follow this  
Followers 0