[{"data":1,"prerenderedAt":408},["ShallowReactive",2],{"content-\u002Fcontents\u002Fweightbiases":3,"surroundPost-\u002Fcontents\u002Fweightbiases":399},{"id":4,"title":5,"body":6,"createdAt":387,"description":72,"draft":388,"extension":389,"meta":390,"navigation":136,"path":391,"seo":392,"stem":393,"tags":394,"thumbnail":397,"updatedAt":387,"__hash__":398},"contents\u002Fcontents\u002Fweightbiases.md","Weights&Biasesで機械学習の実験管理を行う方法",{"type":7,"value":8,"toc":377},"minimark",[9,13,16,27,30,38,41,46,54,57,60,63,66,92,99,102,107,110,113,116,119,155,162,165,176,227,230,233,236,239,307,310,313,316,319,322,326,329,332,352,355,358,361,364,367,370,373],[10,11,12],"p",{},"久々に機械学習記事です。",[10,14,15],{},"今回はローカルの実験管理ツールについてです。",[10,17,18,19,26],{},"ずばり調査したのは ",[20,21,25],"a",{"href":22,"rel":23},"https:\u002F\u002Fwandb.ai\u002Fsite",[24],"nofollow","Weights & Biases"," です。",[10,28,29],{},"なぜ weights&biases かというと元々は Mlflow の利用を考えていたのですが、たまたま Twitter で weights&biases が便利だという話を聞いたので調査してみました。",[10,31,32,33,37],{},"結論から言うと",[34,35,36],"strong",{},"超便利","だったので今後のローカル実験管理は weights&biases で行くことに決めました。",[10,39,40],{},"間違ったとこもあるかもですが、とりあえずわかったことを中心にまとめてみました。",[42,43,45],"h2",{"id":44},"weightsbiases-の使い方","weights&biases の使い方",[10,47,48,53],{},[20,49,52],{"href":50,"rel":51},"https:\u002F\u002Fdocs.wandb.ai\u002Fquickstart",[24],"Quickstart"," の内容をまずは試してみるのをおすすめします。",[10,55,56],{},"weights&biases は利用するのに GitHub か Google で登録が必要がなります。",[10,58,59],{},"ブラウザで登録し、login すると API キーが参照できるのでそれをコピーしておきます。",[10,61,62],{},"続いてはローカルで weights&biases のパッケージである wandb の準備をします。",[10,64,65],{},"コマンドは以下の通り（※ poetry 想定です）",[67,68,73],"pre",{"className":69,"code":70,"language":71,"meta":72,"style":72},"language-bash shiki shiki-themes github-dark","poetry add wandb\n","bash","",[74,75,76],"code",{"__ignoreMap":72},[77,78,81,85,89],"span",{"class":79,"line":80},"line",1,[77,82,84],{"class":83},"svObZ","poetry",[77,86,88],{"class":87},"sU2Wk"," add",[77,90,91],{"class":87}," wandb\n",[10,93,94,95,98],{},"install 後、",[74,96,97],{},"wandb login","をコマンドラインで入力すると、対話形式で API キーを聞かれるので先ほどコピーした値を貼り付けます。",[10,100,101],{},"これでコードで wandb を使用する準備ができました。",[103,104,106],"h3",{"id":105},"各種設定パラメータの管理","各種設定、パラメータの管理",[10,108,109],{},"まずは機械学習に欠かせないパラメータ管理です。",[10,111,112],{},"正直これが使いたくて今回実験管理ツールを探していました。",[10,114,115],{},"実際のコードです。",[10,117,118],{},"一番単純にやる場合は以下のコードのみです",[67,120,124],{"className":121,"code":122,"language":123,"meta":72,"style":72},"language-py shiki shiki-themes github-dark","import wandb\n\nconfigs={\"epoch\":100, \"learning_rate\":0.001}\n\nwandb.init(project=\"test-project\", config=configs)\n","py",[74,125,126,131,138,144,149],{"__ignoreMap":72},[77,127,128],{"class":79,"line":80},[77,129,130],{},"import wandb\n",[77,132,134],{"class":79,"line":133},2,[77,135,137],{"emptyLinePlaceholder":136},true,"\n",[77,139,141],{"class":79,"line":140},3,[77,142,143],{},"configs={\"epoch\":100, \"learning_rate\":0.001}\n",[77,145,147],{"class":79,"line":146},4,[77,148,137],{"emptyLinePlaceholder":136},[77,150,152],{"class":79,"line":151},5,[77,153,154],{},"wandb.init(project=\"test-project\", config=configs)\n",[10,156,157,158,161],{},"wandb をインポートし、",[74,159,160],{},"wandb.init","でプロジェクトに接続（なければ作成）し、新しい実験（実行）の記録が開始されます。\nこの init の段階で記録したいパラメータを渡しておきます。",[10,163,164],{},"wandb ではプロジェクトの下にいくつもの実験が記録される仕組みです。\nこれにより、あるタスクの訓練をいろいろモデルやパラメータを変えて試行錯誤した結果を追跡できます。",[10,166,167,168,171,172,175],{},"私の場合は、後のコード内で",[74,169,170],{},"configs","の値を使いやすくするために",[74,173,174],{},"pydantic","を使用しています。",[67,177,179],{"className":121,"code":178,"language":123,"meta":72,"style":72},"import wandb\nfrom pydantic import BaseSettings\n\nclass Settings(BaseSettings):\n    learning_rate: float = 0.001\n    epoch: int = 100\n\nsettings = Settings()\nwandb.init(project=\"test-project\", config=settings.dict())\n",[74,180,181,185,190,194,199,204,210,215,221],{"__ignoreMap":72},[77,182,183],{"class":79,"line":80},[77,184,130],{},[77,186,187],{"class":79,"line":133},[77,188,189],{},"from pydantic import BaseSettings\n",[77,191,192],{"class":79,"line":140},[77,193,137],{"emptyLinePlaceholder":136},[77,195,196],{"class":79,"line":146},[77,197,198],{},"class Settings(BaseSettings):\n",[77,200,201],{"class":79,"line":151},[77,202,203],{},"    learning_rate: float = 0.001\n",[77,205,207],{"class":79,"line":206},6,[77,208,209],{},"    epoch: int = 100\n",[77,211,213],{"class":79,"line":212},7,[77,214,137],{"emptyLinePlaceholder":136},[77,216,218],{"class":79,"line":217},8,[77,219,220],{},"settings = Settings()\n",[77,222,224],{"class":79,"line":223},9,[77,225,226],{},"wandb.init(project=\"test-project\", config=settings.dict())\n",[103,228,229],{"id":229},"分類系の情報をまとめて作成",[10,231,232],{},"wandb には機械学習ライブラリやフレームワークごとに便利な plot 機能が用意されています。",[10,234,235],{},"今回は scikit-learn で分類問題を学習する際に便利なメソッドを紹介します。",[10,237,238],{},"使い方は超簡単です。モデル学習後に以下の情報をメソッドに入力するだけです。",[67,240,242],{"className":121,"code":241,"language":123,"meta":72,"style":72},"wandb.sklearn.plot_classifier(\n        clf, # 学習済み分類器\n        X_train, # 訓練用データ\n        x_val, # 検証用データ\n        y_train, # 訓練正解データ\n        y_val, # 検証正解データ\n        y_pred, # 検証用データの予測ラベル結果\n        y_proba, # 検証用データの予測確率\n        labels, # 正解ラベルの種類\n        model_name=\"MLP\",\n        feature_names=None,\n    )\n\n",[74,243,244,249,254,259,264,269,274,279,284,289,295,301],{"__ignoreMap":72},[77,245,246],{"class":79,"line":80},[77,247,248],{},"wandb.sklearn.plot_classifier(\n",[77,250,251],{"class":79,"line":133},[77,252,253],{},"        clf, # 学習済み分類器\n",[77,255,256],{"class":79,"line":140},[77,257,258],{},"        X_train, # 訓練用データ\n",[77,260,261],{"class":79,"line":146},[77,262,263],{},"        x_val, # 検証用データ\n",[77,265,266],{"class":79,"line":151},[77,267,268],{},"        y_train, # 訓練正解データ\n",[77,270,271],{"class":79,"line":206},[77,272,273],{},"        y_val, # 検証正解データ\n",[77,275,276],{"class":79,"line":212},[77,277,278],{},"        y_pred, # 検証用データの予測ラベル結果\n",[77,280,281],{"class":79,"line":217},[77,282,283],{},"        y_proba, # 検証用データの予測確率\n",[77,285,286],{"class":79,"line":223},[77,287,288],{},"        labels, # 正解ラベルの種類\n",[77,290,292],{"class":79,"line":291},10,[77,293,294],{},"        model_name=\"MLP\",\n",[77,296,298],{"class":79,"line":297},11,[77,299,300],{},"        feature_names=None,\n",[77,302,304],{"class":79,"line":303},12,[77,305,306],{},"    )\n",[10,308,309],{},"データの準備と学習さえ終わっていればどれも準備は簡単です。",[10,311,312],{},"これを実行すると混同行列、ROC 曲線、Precision Recall、ターゲットクラスの割合、accuracy\u002Ff1\u002Fprecision\u002Frecall の値が全て可視化・保存されます。",[10,314,315],{},"コードもスッキリしてみやすくなります。",[42,317,318],{"id":318},"その他",[10,320,321],{},"まだ調査途中で実際に使ってないものもありますが、メモがてら書いておきます",[103,323,325],{"id":324},"model-保存方法","model 保存方法",[10,327,328],{},"artifact としてファイルなどを保存できるので、そのやり方で学習済みモデルを保存します。",[10,330,331],{},"実際のコードは以下の通りです。",[67,333,335],{"className":121,"code":334,"language":123,"meta":72,"style":72},"art = wandb.Artifact(f\"{model_name}-clf-{wandb.run.id}\", type=\"model\")\nart.add_file(\"path\u002Fto\u002Fmodel\u002F\")\nrun.log_artifact(art)\n",[74,336,337,342,347],{"__ignoreMap":72},[77,338,339],{"class":79,"line":80},[77,340,341],{},"art = wandb.Artifact(f\"{model_name}-clf-{wandb.run.id}\", type=\"model\")\n",[77,343,344],{"class":79,"line":133},[77,345,346],{},"art.add_file(\"path\u002Fto\u002Fmodel\u002F\")\n",[77,348,349],{"class":79,"line":140},[77,350,351],{},"run.log_artifact(art)\n",[10,353,354],{},"ドキュメントをざっと読んだ感じですと、wandb の Artifact は単純にファイルがプロジェクトに紐づけられて保存されているっぽいです。\ntype は何でもいいらしいです。",[10,356,357],{},"使用するときは download メソッドでローカルに落として使います。",[42,359,360],{"id":360},"使ってみた感想",[10,362,363],{},"めちゃめちゃ使いやすくてびっくりです。",[10,365,366],{},"大してドキュメントを読んだわけではないですが、直感的に使えます。",[10,368,369],{},"一番いいのは既存の学習コードをあまり変えなくていいところと、グラフを作成するコートが省けることです。",[10,371,372],{},"今後も引き続き使っていきます。",[374,375,376],"style",{},"html pre.shiki code .svObZ, html code.shiki .svObZ{--shiki-default:#B392F0}html pre.shiki code .sU2Wk, html code.shiki .sU2Wk{--shiki-default:#9ECBFF}html .default .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}html .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}",{"title":72,"searchDepth":133,"depth":133,"links":378},[379,383,386],{"id":44,"depth":133,"text":45,"children":380},[381,382],{"id":105,"depth":140,"text":106},{"id":229,"depth":140,"text":229},{"id":318,"depth":133,"text":318,"children":384},[385],{"id":324,"depth":140,"text":325},{"id":360,"depth":133,"text":360},"2022-08-13",false,"md",{},"\u002Fcontents\u002Fweightbiases",{"title":5,"description":72},"contents\u002Fweightbiases",[395,396],"機械学習","2022","\u002Fimg\u002Ftwitter-card.png","iM8QSc4P1CiPUiDgXCVk4hFoeodmd4Jcf_O1pVDKuC8",[400,404],{"title":401,"path":402,"stem":403,"children":-1},"【VueとNuxt】v-on, v-if, v-bind, v-modelの使い方紹介","\u002Fcontents\u002Fvue_begin2","contents\u002Fvue_begin2",{"title":405,"path":406,"stem":407,"children":-1},"What makes a harness a harness: necessary and sufficient conditions for an agent harness","\u002Fcontents\u002Fwhat-makes-a-harness","contents\u002Fwhat-makes-a-harness",1782344032470]