プログラマでありたい

おっさんになっても、プログラマでありつづけたい

Mahoutのサンプルプログラミング

プログラム

　ちょっと空いた時間にTasteの後を継ぐオープンソースのリコメンドエンジンのMahoutを触ってみました。まだ使いこなせていないですが、かなり面白そうなアプリです。
　今回はLivedoor Clipsのデータを使って、URLに対してお勧めのタグとタグに対してお勧めのタグを出してみました。タグ情報を使いやすいように整形しているので、詳しくは「livedoor clipsのデータを少しだけ眺めてみた。」を参照してください。


import java.io.File;
import java.io.FileNotFoundException;
import java.util.List;

import org.apache.mahout.cf.taste.common.TasteException;
import org.apache.mahout.cf.taste.impl.model.file.FileDataModel;
import org.apache.mahout.cf.taste.impl.neighborhood.NearestNUserNeighborhood;
import org.apache.mahout.cf.taste.impl.recommender.CachingRecommender;
import org.apache.mahout.cf.taste.impl.recommender.GenericItemBasedRecommender;
import org.apache.mahout.cf.taste.impl.recommender.GenericUserBasedRecommender;
import org.apache.mahout.cf.taste.impl.similarity.AveragingPreferenceInferrer;
import org.apache.mahout.cf.taste.impl.similarity.LogLikelihoodSimilarity;
import org.apache.mahout.cf.taste.impl.similarity.PearsonCorrelationSimilarity;
import org.apache.mahout.cf.taste.model.DataModel;
import org.apache.mahout.cf.taste.model.Item;
import org.apache.mahout.cf.taste.model.User;
import org.apache.mahout.cf.taste.neighborhood.UserNeighborhood;
import org.apache.mahout.cf.taste.recommender.ItemBasedRecommender;
import org.apache.mahout.cf.taste.recommender.RecommendedItem;
import org.apache.mahout.cf.taste.recommender.Recommender;
import org.apache.mahout.cf.taste.similarity.ItemSimilarity;
import org.apache.mahout.cf.taste.similarity.UserSimilarity;

public class Recommend {

	/**
	 * @param args
	 */
	public static void main(String[] args) {
		// TODO 自動生成されたメソッド・スタブ
		try {
			DataModel model = new FileDataModel(new File("/tmp/clips.txt"));
			Recommend.itemBaseRecommender(model);
			Recommend.userBaseRecommender(model);

		} catch (FileNotFoundException e) {
			e.printStackTrace();
		}
	}

	private static void itemBaseRecommender(DataModel model) {
		try {
			ItemSimilarity itemSimilarity = new LogLikelihoodSimilarity(model);
			System.out.println("ロード完了");
			ItemBasedRecommender itemRecommender =
				  new GenericItemBasedRecommender(model, itemSimilarity);
			for (Item item : model.getItems()) {
				System.out.println("ITEM=" + model.getItem(item.getID()));
				List<RecommendedItem> recommendations =
					  itemRecommender.mostSimilarItems(item.getID(), 10);
				for (RecommendedItem recommend : recommendations) {
					System.out.println("Recommend=" + recommend.getItem());
				}
			}
		} catch (TasteException e) {
			e.printStackTrace();
		}

		}

	private static void userBaseRecommender(DataModel model) {
		try {
			UserSimilarity userSimilarity = new PearsonCorrelationSimilarity(model);
			userSimilarity.setPreferenceInferrer(new AveragingPreferenceInferrer(model));
			System.out.println("ロード完了");
			UserNeighborhood neighborhood =
				  new NearestNUserNeighborhood(3, userSimilarity, model);
			System.out.println("UserNeighborhood完了");
			Recommender recommender =
				  new GenericUserBasedRecommender(model, neighborhood, userSimilarity);
			Recommender cachingRecommender = new CachingRecommender(recommender);
			System.out.println("Recommender完了");
			for (User user : model.getUsers()) {
				System.out.println("User=" + model.getUser(user.getID()));
				List<RecommendedItem> recommendations =
					cachingRecommender.recommend(user.getID(), 10);
				for (RecommendedItem recommend : recommendations) {
					System.out.println(recommend.getItem());
				}
			}
/**
			List<RecommendedItem> recommendations =
					  cachingRecommender.recommend("http://d.hatena.ne.jp/naoya/", 10);
			//		userSimilarity.userSimilarity("http://clip.livedoor.com/", "http://b.hatena.ne.jp/")
			for (RecommendedItem recommend : recommendations) {
				System.out.println(recommend.getItem());
			}
*/
		} catch (TasteException e) {
			e.printStackTrace();
		}
	}
}


実行すると、こんな感じです。

09/07/08 11:05:05 INFO file.FileDataModel: Creating FileDataModel for file \tmp\clips.txt
ロード完了
09/07/08 11:05:05 INFO file.FileDataModel: Reading file info...
09/07/08 11:05:06 INFO file.FileDataModel: Processed 100000 lines
09/07/08 11:05:07 INFO file.FileDataModel: Processed 200000 lines
09/07/08 11:05:07 INFO file.FileDataModel: Processed 300000 lines
09/07/08 11:05:08 INFO file.FileDataModel: Processed 400000 lines
09/07/08 11:05:08 INFO file.FileDataModel: Processed 500000 lines
09/07/08 11:05:09 INFO file.FileDataModel: Processed 600000 lines
09/07/08 11:05:10 INFO file.FileDataModel: Processed 700000 lines
09/07/08 11:05:11 INFO file.FileDataModel: Processed 800000 lines
09/07/08 11:05:11 INFO file.FileDataModel: Processed 900000 lines
09/07/08 11:05:12 INFO file.FileDataModel: Processed 1000000 lines
09/07/08 11:05:13 INFO file.FileDataModel: Processed 1100000 lines
09/07/08 11:05:13 INFO file.FileDataModel: Processed 1200000 lines
09/07/08 11:05:14 INFO file.FileDataModel: Processed 1300000 lines
09/07/08 11:05:15 INFO file.FileDataModel: Processed 1400000 lines
09/07/08 11:05:15 INFO file.FileDataModel: Processed 1500000 lines
09/07/08 11:05:15 INFO file.FileDataModel: Read lines: 1507063
09/07/08 11:05:16 INFO model.GenericDataModel: Processed 10000 users
09/07/08 11:05:16 INFO model.GenericDataModel: Processed 20000 users
09/07/08 11:05:16 INFO model.GenericDataModel: Processed 30000 users
09/07/08 11:05:16 INFO model.GenericDataModel: Processed 40000 users
09/07/08 11:05:16 INFO model.GenericDataModel: Processed 50000 users
09/07/08 11:05:16 INFO model.GenericDataModel: Processed 60000 users
09/07/08 11:05:16 INFO model.GenericDataModel: Processed 70000 users
09/07/08 11:05:17 INFO model.GenericDataModel: Processed 80000 users
09/07/08 11:05:17 INFO model.GenericDataModel: Processed 90000 users
09/07/08 11:05:17 INFO model.GenericDataModel: Processed 100000 users
09/07/08 11:05:17 INFO model.GenericDataModel: Processed 110000 users
09/07/08 11:05:18 INFO model.GenericDataModel: Processed 120000 users
09/07/08 11:05:18 INFO model.GenericDataModel: Processed 130000 users
09/07/08 11:05:18 INFO model.GenericDataModel: Processed 140000 users
09/07/08 11:05:18 INFO model.GenericDataModel: Processed 150000 users
09/07/08 11:05:18 INFO model.GenericDataModel: Processed 160000 users
09/07/08 11:05:18 INFO model.GenericDataModel: Processed 170000 users
09/07/08 11:05:18 INFO model.GenericDataModel: Processed 180000 users
09/07/08 11:05:18 INFO model.GenericDataModel: Processed 190000 users
09/07/08 11:05:19 INFO model.GenericDataModel: Processed 200000 users
〜中略〜
ITEM=Item[id:!!（゜□゜ψ）ψ]
Recommend=Item[id:テレビ論]
Recommend=Item[id:ややうけ]
Recommend=Item[id:(ﾟ∀ﾟ)=３]
Recommend=Itemid:cine
Recommend=Item[id:連想]
Recommend=Item[id:!!!]
Recommend=Item[id:ジャッキー・チェン]
Recommend=Itemid:meta
Recommend=Itemid:cinema
Recommend=Item[id:爆笑した]
ITEM=Item[id:!dankogai]
Recommend=Item[id:アルファギーク本]
Recommend=Item[id:アルファギーク(笑)]
Recommend=Item[id:世界の矢野]
Recommend=Item[id:ﾟДﾟ)]
Recommend=Itemid:april
Recommend=Item[id:子飼弾]
Recommend=Item[id:*ネタ]
Recommend=Item[id:魔法少女はてなちゃん]
Recommend=Item[id:はてな的なるもの]
Recommend=Item[id:大喜利]
ITEM=Item[id:!gtd]
Recommend=Item[id:資料(いつか使う)]
Recommend=Item[id:[!gtd]
Recommend=Item[id:資料(いつか使う)]]
Recommend=Item[id:*mac]
Recommend=Item[id:!gtd資料(いつか使う)]
Recommend=Item[id:作業効率]
Recommend=Itemid:itunes
Recommend=Itemid:mac
Recommend=Itemid:iTunes
Recommend=Itemid:INTERNET
〜中略〜
User=User[id:http://d.hatena.ne.jp/jkondo]
Item[id:はてな]
Item[id:格]
Item[id:エントリ]
Item[id:フリーク]
Item[id:はてブ]
Item[id:ネタ]
Item[id:流行]
Itemid:music
Item[id:ヘンテコ]
Item[id:定着]
〜略〜

　元データにノイズがかなり含まれているのですが、それを除けばすぐに使えそうです。次はデータソースをファイルではなくDBに変更するのと、永続化を試してみようかと思います。