CareerCup

Assuming that File-1 and File-2 have no duplicates within themselves? And memory limitation is not a issue:
1. Iterate through File-2 adding each string to a HashMap and writing each string to File-3
2. Iterate through File-1, check if each string is present in the HashMap, if its not then write the string to File-3.
If memory is an issue, you could use a memory-mapped-file to store the HashMap strings.

This problem is related to the time and space trade off.

1. Space/Memory is the issue, time is not a prime concern.

Steps:
a. Sort File -1 and File-2 using External sort.
b. Now merge the sorted file File 3.

Almost no extra space required. But it requires O(nlgn+mlgm) time.

2. If the time is the issue and memory is not factor at all.

Steps:
a. Create hash map that map keys to string.
b. Start with either File-1 or File-2. Pick a string and generate a hash key.
c. Store the string in the hash map with the key.
d. Write the string in the file.
e. Pick another string do the same.
f. If there is collision in the hash table, then compare the current string the existing string in the hash table. If they are same, then discard the current string.
g. If the current string and the string in the table is different, then add this current string using chaining method to the hash table. Write down the current string into the File -3.


Complexity:

Time complexity : O(n+m)

Space Complexity: If we take a hash table that can hold O(m) number of keys. Then, if there are lots of duplicates, then the size would be O(m). In worst case, if there is no duplicates, the the total entry in the hash table is O(n+m)

if not concerned about space add the strings to a set which will eliminate duplicates and as well they will be merged.

Memory is the issue. Assume md5sum of each string is 4 bytes and no duplicate string, the set would be at least 6,000,000 x 4 bytes.

these days, computer has much memory more than 4~8GB just 6,000,000 strings whose size is 1024. it is just 6,000,000 * 1024 = 6GB, just read at once, and make tree and merge -_-;

creating a suffix tree or trie out of file 1 and then traversing through file 2 and comparing with trie created

Assign numerical values(hash values) to each string using 26-radix. For example, cat = 3*(26 pow 2)+1*(26 pow 1)+20*(26 pow 0)
That leaves us with strings which have different numerical value or same numerical value but are either exactly same strings OR different.To compare two strings, first check if their numerical value is same. Only if its same, match letter by letter.
Now sort smaller file and for each word in larger file, do a binary search in smaller file(height of smaller file is lesser and sorting it also takes lesser time).

1 calculate hash-code of each string in the file-1 and use it as key to check whether there already a element in it, if so, check the file-3 whether the string has been saved in it.
2 If the string has not been saved in file-3, save it, otherwise,do nothing.
3 repeat step 1 and 2 for file-2.

If you work with php,

1. Get the file contents into array
2. Use array merge
3. Then use arrray_unique
4 Then write back the new array to file