Experimenting with markov chains

gistfile1.txt

All bribery and dwell in Jerusalem, seeking ing the LORD of a garment; And he saith, To the counsel of his throne, saying, this second time, and in his place. Trust in the interpretation; the tenths and there remained among all nations bow down the earth, which now I will but thou shalt drive out of heart, My praise unto the atonement, whereby they are not drink it. So thy wonders have wounded him that he to him the adoption, the son of their order, and the children of gold in the mountain on this thing by their nakedness of waters, that is girded on us, it came unto me, and spoiled them. Let the land of and they put evil of the daughter of Israel that told the heritage unto them, and escape out of For behold king of their dwelling among great nation in unto them, They envied him. He hath kept close, and he shall speak. Consume in the children of the LORD will slay me. When thou shalt put all thy dishonest gain. For we all the but whosoever believeth all his name Apollyon. One land into the inheritance in the women, that ye shall it was translated. Yea rather, that wherein thou hast thou shalt ask a candle, and said, How great a stone, silver five thousand men, and said the righteous, And he made thee with the thirty and the lust of the faith without further question, according to the for lack of the hand of the foundation of all his and compass me daily, consulting alway ready. He said unto the city, shake out of Israel journeyed from twenty and tongue, it in the sons of them that went and the whole multitude gathered their Of Mahli and then took two rows of the anointing shall bring evil in the congregation, the Jews had the river, How are buried in Judea without our God; Having the head of his host, and Abraham said, Lord, receive you, saith the dead. There they may be charged; that thou dost thou offer with bread. A Psalm 98 A merchant in his neighbour; city with brass. And the earth, bless thee; Leave off in pieces enough for the valley of the chief of the country, being assembled together the strength from the Moabitess, her kindred of the Hittites, and let the resurrection of Cephas, or the cities of hosts, even Epher, and he joined unto them, O God, and horses, neither was numbered of the tribe of Amariah, Malluch, Shemariah. Of their tents, and let her friendship; and Izhar, and get thee into thine own people, whom Tahpenes bare rule every wise man that were the LORD to his disciples and Zilthai, captains of the marriage. And the sacrifice the end upon him; for the false witness in God, to death, even that wicked that therein. When thou shalt not answer. The captain of a vanity and all the tabernacle; When the place called it thou shalt not thy land that created now, and they put on. As for ever. And she turned back, dwell among thy people and said unto me, Solomon raised; for the house of the holy things, which was upon us from building of the Jordan before Joshua; and my son. And he said unto him to the LORD thy vows. Psalm of the arrow shot at thy the feet that they wrote in great God is become of the multitude of clay of heaven. He fought against thee, that, denying the people and ever. Amen. Cursed he shall lay upon them, Verily I will of Jehoshaphat slept with fleshly lusts, which is a staff shall hold upon thee, and all his servant, and the meat offering, which upon them repaired Hananiah the son of them and did boldly take but I have sent messengers, and it shall be and justly according to those which say, I will instruct and their pillars and calamities of them that at all the sons shall a drink water, I did All the house of gold for thy servants, which had taken up also the husbandmen said unto Gibeon. Now when it came and no glory and told abroad, and to his two cubits; and Thou like the earth, and there shall be found the hundred and said, Not forsaking in the stranger in Jerusalem, building that day, and brought to him in a wife hath promised to them they say, not nigh at end, Thus shall bring in Shiloh. Then his brother. Did not the LORD all that she shewed him instantly, saying, Thus shalt bring an ensign upon the workers of thine house of mine understanding exceeding dreadful, whose oath that have made ten acres of ungodly and drink offering of the LORD answered him an upper Cilicia. Then the house of Bethulia. And they asked him in the service of the congregation of man, and sincerely, in the word of burning. If it fifty righteous thing heretofore. And the land. Therefore said unto thee, whited for he goats, with speed, haste, and they answered them, that the tabernacle of these years will return to ride upon a certain of angels, and twenty thousand, and be half of Nathan the priest's daughter of seasons and unto one of an inhabitant. They made an offering made that they should see the way side, when he coupled the man with her widowhood, save the land of badgers skins. And I, whither the evil in my father, or in the nation, among you. If therefore ye shall be fulfilled. And David at that thy father Abraham gave strength he shall see upon him, one he brought forth the dust of Egypt. And if thou shalt say unto the cause thee an altar also the third day with their garments. 

markov.js

function normalize(obj) {
    var total = 0, result = {}, k;
    for (k in obj) total += obj[k];
    for (k in obj) result[k] = obj[k] / total;
    return result;
}

function build(input, freq) {
    var words = input.split(/\s+/)
        .filter(function (w) {
            return /^[a-z0-9\.,;'\-]+$/i.test(w);
        });
    var max = words.length - 1;
 
    function add(a, b) {
        if (!(a in freq))    freq[a] = {};
        if (!(b in freq[a])) freq[a][b] = 0;
        freq[a][b]++;
    }
    
    words.forEach(function (w, i) {
        if (i < max) add(w, words[i + 1]);
    });

    return freq;
}

function next(word, freq) {
    var i = Math.random();
    var table = normalize(freq[word]);
    var total = 0;
    var k;

    for (k in table) { 
        total += table[k];
        if (total >= i) return k;
    }
        
    return k;
}

var ls = require('fs').readdirSync;
var freq = {};
ls('.').forEach(function (file) {
    var txt = require('fs').readFileSync(file, 'utf-8');
    build(txt, freq);
});

function starts(word, n) {
    var result = [];

    while (n--) {
        result.push(word);
        word = next(word, freq);
    }

    return result.join(' ');
}

/*global process: true */
process.stdout.write(starts('All', 1000) + '\n');