[{"data":1,"prerenderedAt":1175},["ShallowReactive",2],{"content-\u002Fcontents\u002Ftensorboard":3,"surroundPost-\u002Fcontents\u002Ftensorboard":1166},{"id":4,"title":5,"body":6,"createdAt":1153,"description":1154,"draft":1155,"extension":1156,"meta":1157,"navigation":297,"path":1158,"seo":1159,"stem":1160,"tags":1161,"thumbnail":1164,"updatedAt":1153,"__hash__":1165},"contents\u002Fcontents\u002Ftensorboard.md","PyTorchでTensorBoardを使う方法",{"type":7,"value":8,"toc":1142},"minimark",[9,13,17,20,28,31,35,38,41,57,61,64,67,70,73,76,79,82,85,104,107,111,114,117,120,123,126,133,142,145,148,156,167,170,200,203,206,244,255,262,268,271,274,391,398,422,425,436,439,445,449,464,468,471,475,478,484,497,504,507,511,514,521,527,530,539,542,548,992,995,999,1005,1014,1017,1021,1024,1027,1031,1034,1038,1045,1048,1051,1055,1058,1064,1067,1070,1089,1092,1095,1098,1101,1104,1110,1116,1119,1122,1138],[10,11,12],"h2",{"id":12},"はじめに",[14,15,16],"p",{},"どうも。おい丸です。",[14,18,19],{},"今回は久々に技術系の記事を書きました。",[14,21,22,23,27],{},"内容は",[24,25,26],"strong",{},"Pytorch での TensorBoard の使い方","になります。",[14,29,30],{},"ちょっと前に書いた Titanic 以来の機械学習系の記事です。",[10,32,34],{"id":33},"tensorboard-とは","TensorBoard とは",[14,36,37],{},"TensorBoard は、TensorFlow に付属する可視化機能です。\nコマンドで起動すると、ブラウザで可視化用のダッシュボードを利用して各種結果をみることができます。",[14,39,40],{},"機械学習でよく可視化するエポックごとの loss や accuracy をはじめとした値を簡単に可視化することができます。",[14,42,43,44,48,49,52,53,56],{},"また、loss などのスカラーだけでなく、",[45,46,47],"code",{},"Image","や",[45,50,51],{},"audio","、",[45,54,55],{},"text","などに関しても簡単に可視化が可能です。",[10,58,60],{"id":59},"なぜ-tensorboard","なぜ TensorBoard？",[14,62,63],{},"Python には matplotlib という可視化のライブラリがあるのですが、描画の設定などが若干面倒です。",[14,65,66],{},"私の場合は matplotlib が苦手で、いつになっても書き方を覚えられず、毎回ググってました。",[14,68,69],{},"一方で、TensorBoard はグラフの表示部分は自動でやってくれるので覚えることが少なく、コードの記述量も少なくて済みます。",[14,71,72],{},"加えて、データはファイルに書き出されるので、学習終了後にいつでも結果をみることができます。",[14,74,75],{},"さらに書き出されたデータはリアルタイムでブラウザ上のグラフに反映されます。",[14,77,78],{},"また、グラフは GUI 操作でダウンロードすることも可能です。",[14,80,81],{},"ファイルの書き出しやグラフのダウンロードのコードを書かなくてもいいのはほんとうに便利です。",[14,83,84],{},"私が感じた TensorBoard の利点をまとめると以下になります。",[86,87,88,92,95,98,101],"ul",{},[89,90,91],"li",{},"簡単にデータをグラフ表示可能",[89,93,94],{},"グラフの保存が GUI で容易にできる",[89,96,97],{},"データは外部ファイルに書き出されるので、結果の管理に使える",[89,99,100],{},"学習経過をリアルタイムに監視できる",[89,102,103],{},"jupyter notebook からダッシュボードをひらける",[14,105,106],{},"他のものでも代替できるとは思うのですが、TensorBoard は学習コストが少なく手っ取り早く使えるのもいいです。",[10,108,110],{"id":109},"pytorch-で-tensorboard-を使ってみる","Pytorch で TensorBoard を使ってみる",[14,112,113],{},"上記のように便利な TensorBoard ですが、TensorBoard は TensorFlow じゃなきゃ使えないと思っていました。",[14,115,116],{},"しかしなんと Pytorch でも使えるんです！",[14,118,119],{},"私はディープラーニング系は Pytorch でずっと書いてきたので、Pytorch で使えるのはありがたいです。",[14,121,122],{},"それではさっそく Pytorch と TensorBoard が使える環境構築からやっていきます。",[14,124,125],{},"今回は Dcoker で環境構築を行います。",[14,127,128,129,132],{},"以下が今回使用した",[45,130,131],{},"Dockerfile","です。",[134,135,139],"pre",{"className":136,"code":138,"language":55},[137],"language-text","FROM nvcr.io\u002Fnvidia\u002Fpytorch:19.08-py3\n\nRUN apt-get clean && apt-get update\n\nRUN pip install msgpack==0.5.6\nRUN pip install pillow tensorboard matplotlib tensorflow future torch torchvision --ignore-installed\nRUN pip install wrapt==1.10.0 --ignore-installed\n\nWORKDIR \u002Fworkspace\u002Fwork\n\nCMD \u002Fbin\u002Fbash\n",[45,140,138],{"__ignoreMap":141},"",[14,143,144],{},"(Dockerfile に慣れてないので微妙な書き方かもしれないです)",[14,146,147],{},"ちなみに Pytorch で TensorBoard を利用するのに必要なのは以下のバージョンになります。",[86,149,150,153],{},[89,151,152],{},"pytorch >= 1.1.0",[89,154,155],{},"tensorboard >= 1.14",[14,157,158,159,166],{},"（参考：",[160,161,165],"a",{"href":162,"rel":163},"https:\u002F\u002Fwww.endtoend.ai\u002Fcontents\u002Fpytorch-tensorboard\u002F",[164],"nofollow","Using TensorBoard with PyTorch 1.1 or 1.2","）",[14,168,169],{},"これをてきとうなフォルダに置いて、同じフォルダ内で以下のコマンドを実行します。",[134,171,175],{"className":172,"code":173,"language":174,"meta":141,"style":141},"language-bash shiki shiki-themes github-dark","docker build -t tensorboard .\n","bash",[45,176,177],{"__ignoreMap":141},[178,179,182,186,190,194,197],"span",{"class":180,"line":181},"line",1,[178,183,185],{"class":184},"svObZ","docker",[178,187,189],{"class":188},"sU2Wk"," build",[178,191,193],{"class":192},"sDLfK"," -t",[178,195,196],{"class":188}," tensorboard",[178,198,199],{"class":188}," .\n",[14,201,202],{},"これで今回の作業用 Docker image が作成されます。\nイメージサイズが 10GB 近くなるので注意です。",[14,204,205],{},"イメージ作成後は以下のコマンドで、コンテナを起動します。",[134,207,209],{"className":172,"code":208,"language":174,"meta":141,"style":141},"docker run --runtime=nvidia -it -v \u002Fpath\u002Fto\u002Fdir:\u002Fworkspace\u002Fwork -p 8888:8888 -p 6006:6006 tensorboard\n",[45,210,211],{"__ignoreMap":141},[178,212,213,215,218,221,224,227,230,233,236,238,241],{"class":180,"line":181},[178,214,185],{"class":184},[178,216,217],{"class":188}," run",[178,219,220],{"class":192}," --runtime=nvidia",[178,222,223],{"class":192}," -it",[178,225,226],{"class":192}," -v",[178,228,229],{"class":188}," \u002Fpath\u002Fto\u002Fdir:\u002Fworkspace\u002Fwork",[178,231,232],{"class":192}," -p",[178,234,235],{"class":188}," 8888:8888",[178,237,232],{"class":192},[178,239,240],{"class":188}," 6006:6006",[178,242,243],{"class":188}," tensorboard\n",[14,245,246,247,250,251,254],{},"ちょっと長いんですが、やってることとしてはまず GPU が使えるように",[45,248,249],{},"--runtime=nvidia","を指定します。",[252,253],"br",{},"\n（※ nvidia-docker のインストールについては今回は触れません）",[14,256,257,258,261],{},"次に、",[45,259,260],{},"-v \u002Fpath\u002Fto\u002Fdir:\u002Fworkspace\u002Fwork","でローカルの作業フォルダをコンテナ内にマウントします。",[14,263,264,267],{},[45,265,266],{},"-p 8888:8888 -p 6006:6006","は jupyter notebook と TensorBoard を動かすためのポートフォワードになります。",[14,269,270],{},"上記の docker run コマンドを実行するとコンテナ内に入れると思います。",[14,272,273],{},"それでは、簡単なコードで TensorBoard がどんなものか、どのように使えるか確認したいと思います。",[134,275,279],{"className":276,"code":277,"language":278,"meta":141,"style":141},"language-py shiki shiki-themes github-dark","from torch.utils.tensorboard import SummaryWriter\nimport numpy as np\n\n# log出力用のディレクトリを指定\nlog_path = \".\u002Flog\"\n\n# writerオブジェクト生成。パスを引数に指定しない場合は.\u002Fruns\u002F配下に出力される\nwriter = SummaryWriter(log_path)\n\nx = np.array(range(1,10))\ny = x * x\n\n# xとyの値を記録していく\nfor i in range(len(x)):\n    writer.add_scalar(\"x\", x[i], i)\n    writer.add_scalar(\"y\", y[i], i)\n\n# writerを閉じる\nwriter.close()\n","py",[45,280,281,286,292,299,305,311,316,322,328,333,339,345,350,356,362,368,374,379,385],{"__ignoreMap":141},[178,282,283],{"class":180,"line":181},[178,284,285],{},"from torch.utils.tensorboard import SummaryWriter\n",[178,287,289],{"class":180,"line":288},2,[178,290,291],{},"import numpy as np\n",[178,293,295],{"class":180,"line":294},3,[178,296,298],{"emptyLinePlaceholder":297},true,"\n",[178,300,302],{"class":180,"line":301},4,[178,303,304],{},"# log出力用のディレクトリを指定\n",[178,306,308],{"class":180,"line":307},5,[178,309,310],{},"log_path = \".\u002Flog\"\n",[178,312,314],{"class":180,"line":313},6,[178,315,298],{"emptyLinePlaceholder":297},[178,317,319],{"class":180,"line":318},7,[178,320,321],{},"# writerオブジェクト生成。パスを引数に指定しない場合は.\u002Fruns\u002F配下に出力される\n",[178,323,325],{"class":180,"line":324},8,[178,326,327],{},"writer = SummaryWriter(log_path)\n",[178,329,331],{"class":180,"line":330},9,[178,332,298],{"emptyLinePlaceholder":297},[178,334,336],{"class":180,"line":335},10,[178,337,338],{},"x = np.array(range(1,10))\n",[178,340,342],{"class":180,"line":341},11,[178,343,344],{},"y = x * x\n",[178,346,348],{"class":180,"line":347},12,[178,349,298],{"emptyLinePlaceholder":297},[178,351,353],{"class":180,"line":352},13,[178,354,355],{},"# xとyの値を記録していく\n",[178,357,359],{"class":180,"line":358},14,[178,360,361],{},"for i in range(len(x)):\n",[178,363,365],{"class":180,"line":364},15,[178,366,367],{},"    writer.add_scalar(\"x\", x[i], i)\n",[178,369,371],{"class":180,"line":370},16,[178,372,373],{},"    writer.add_scalar(\"y\", y[i], i)\n",[178,375,377],{"class":180,"line":376},17,[178,378,298],{"emptyLinePlaceholder":297},[178,380,382],{"class":180,"line":381},18,[178,383,384],{},"# writerを閉じる\n",[178,386,388],{"class":180,"line":387},19,[178,389,390],{},"writer.close()\n",[14,392,393,394,397],{},"Pytorch で TensorBoard を使うには",[45,395,396],{},"from torch.utils.tensorboard import SummaryWriter","\nをインポートします。",[14,399,400,401,404,405,408,409,412,413,415,416,421],{},"コードのやってることは単純で、",[45,402,403],{},"i","ごとの",[45,406,407],{},"x","と",[45,410,411],{},"y","の値をそれぞれ別のグラフとして記録するというものです。",[252,414],{},"\n詳しい引数などは",[160,417,420],{"href":418,"rel":419},"https:\u002F\u002Fpytorch.org\u002Fdocs\u002Fstable\u002Ftensorboard.html",[164],"公式サイト","を参照ください。",[14,423,424],{},"基本的にタグ、縦軸、横軸のように直感的な引数で簡単です。",[14,426,427,428,431,432,435],{},"上記コードを実行すると、",[45,429,430],{},"log","配下に",[45,433,434],{},"events.out.tfevents.1567930327.d6edca4d0003.82","のようなファイルが生成されます。",[14,437,438],{},"それでは TensorBoard で出力を確認してみましょう。",[14,440,441,442,444],{},"なんと今回の環境だと以下のように jupyter notebook から TensorBoard をひらけます！",[252,443],{},"\nマジ便利です。",[446,447],"img",{"alt":430,"img-src":448},"\u002Fimg\u002Ftensorboard\u002Flog.png",[14,450,451,452,454,455,457,458,460,461,463],{},"右上の New のプルダウンを押下すると、Tensorboard という項目があるのでクリックします。",[252,453],{},"\nCurrent か Custom ディレクトリの選択肢がでるので、今回は Custom を選択。",[252,456],{},"\nlog 用のディレクトリを聞かれるのでパスを入力します。",[252,459],{},"\n今回は",[45,462,430],{},"になるので以下の画像のように入力して ok を押します。",[446,465],{"alt":466,"img-src":467},"log2","\u002Fimg\u002Ftensorboard\u002Flog2.png",[14,469,470],{},"すると、別タブで以下のような TensorBoard のダッシュボードが表示されます。",[446,472],{"alt":473,"img-src":474},"test1","\u002Fimg\u002Ftensorboard\u002Ftest1.png",[14,476,477],{},"いい感じに表示されてます！",[14,479,480,481,483],{},"jupyter を介さずにターミナルから以下のコマンドでひらくこともできます。",[252,482],{},"\nあとから結果だけ確認したい場合に便利です。",[134,485,487],{"className":172,"code":486,"language":174,"meta":141,"style":141},"tensorboard --logdir=log\n",[45,488,489],{"__ignoreMap":141},[178,490,491,494],{"class":180,"line":181},[178,492,493],{"class":184},"tensorboard",[178,495,496],{"class":192}," --logdir=log\n",[14,498,499,500,503],{},"入力後、ブラウザで",[45,501,502],{},"localhost:6006","にアクセスすると以下のようにグラフが表示されます。",[14,505,506],{},"このように、TensorBoard を使うと簡単かつキレイにグラフを作成することができます。",[10,508,510],{"id":509},"tensorboard-で-dcgan-の学習過程を可視化","TensorBoard で DCGAN の学習過程を可視化",[14,512,513],{},"さて、実際にディープラーニングで TensorBoard を使ってみましょう。",[14,515,516,517,520],{},"TensorBoard をディープラーニングで試してみる題材として",[24,518,519],{},"DCGAN","の Pytorch 実装を使います。",[14,522,523,524,526],{},"なぜ DCGAN かというと、GAN の特に古いものは学習が難しく、トライアンドエラーが必要になります。",[252,525],{},"\nloss の値を監視して学習の経過をみるのに TensorBoard が向いてると思ったからです。",[14,528,529],{},"DCGAN の Pytorch 実装は Pytorch の公式サイトにのっているので、そちらを参考に TensorBoard 部分を追記しました。",[14,531,532,533,538],{},"(参考：",[160,534,537],{"href":535,"rel":536},"https:\u002F\u002Fpytorch.org\u002Ftutorials\u002Fbeginner\u002Fdcgan_faces_tutorial.html",[164],"DCGAN TUTORIAL",")",[14,540,541],{},"データは celebA を利用しました。",[14,543,544,545,547],{},"以下がコードになります。TensorBoard 用に変更した部分を抜粋しました。",[252,546],{},"\nそれ以外は上記リンクの実装とほぼ同様なので省略してます。",[134,549,551],{"className":276,"code":550,"language":278,"meta":141,"style":141},"from torch.utils.tensorboard import SummaryWriter\nfrom datetime import datetime\n\n# log用フォルダを毎回生成\nnow = datetime.now()\nlog_path = \".\u002Flog\u002F\" + now.strftime(\"%Y%m%d-%H%M%S\") + \"\u002F\"\nprint(log_path)\n\nimg_list = []\nG_losses = np.array([])\nD_losses = np.array([])\niters = 0\n\n# 学習のループ\nfor epoch in range(n_epoch):\n    for itr, data in enumerate(dataloader):\n        real_image = data[0].to(device)     # 元画像\n        sample_size = real_image.size(0)    # 画像枚数\n        noise = torch.randn(sample_size, nz, 1, 1, device=device)   # 正規分布からノイズを生成\n\n        real_target = torch.full((sample_size,), 1., device=device)\n        fake_target = torch.full((sample_size,), 0., device=device)\n\n        # 識別器Dの更新\n        netD.zero_grad()    # 勾配の初期化\n\n        output = netD(real_image)   # 識別器Dで元画像に対する識別信号を出力\n        errD_real = criterion(output, real_target)  # 元画像に対する識別信号の損失値\n\n        D_x = output.mean().item()\n\n        fake_image = netG(noise)    # 生成器Gでノイズから贋作画像を生成\n\n        output = netD(fake_image.detach())  # 識別器Dで元画像に対する識別信号を出力\n        errD_fake = criterion(output, fake_target)  # 贋作画像に対する識別信号の損失値\n        D_G_z1 = output.mean().item()\n\n        errD = errD_real + errD_fake    # 識別器Dの全体の損失\n        errD.backward()    # 誤差逆伝播\n        optimizerD.step()   # Dのパラメーターを更新\n\n        # 生成器Gの更新\n        netG.zero_grad()    # 勾配の初期化\n\n        output = netD(fake_image)   # 更新した識別器Dで改めて贋作画像に対する識別信号を出力\n        errG = criterion(output, real_target)\n\n        errG.backward()     # 誤差逆伝播\n        D_G_z2 = output.mean().item()\n\n        optimizerG.step()   # Gのパラメータを更新\n\n        if itr % display_interval == 0:\n            print('[{}\u002F{}][{}\u002F{}] Loss_D: {:.3f} Loss_G: {:.3f} D(x): {:.3f} D(G(z)): {:.3f}\u002F{:.3f}'\n                  .format(epoch + 1, n_epoch,\n                          itr + 1, len(dataloader),\n                          errD.item(), errG.item(), D_x, D_G_z1, D_G_z2))\n        if epoch == 0 and itr == 0:     # 初回に元画像を保存する\n            vutils.save_image(real_image, '{}\u002Freal_samples.png'.format(outf),\n                              normalize=True, nrow=10)\n\n        # Save Losses for plotting later\n        G_losses = np.append(G_losses,errG.item())\n        D_losses = np.append(D_losses,errD.item())\n\n        # Tensorboard用のデータ\n        writer.add_scalar(\"G_loss\",errG.item(),iters)\n        writer.add_scalar(\"D_loss\",errD.item(),iters)\n        iters += 1\n\n    if epoch % 10 == 0:\n        fake_image = netG(fixed_noise)\n        writer.add_image(\"fake_image\",vutils.make_grid(fake_image), epoch)\n\n    # モデルの保存\n    if (epoch + 1) % 50 == 0:   # 50エポックごとにモデルを保存する\n        torch.save(netG.state_dict(), '{}\u002FnetG_epoch_{}.pth'.format(outf, epoch + 1))\n        torch.save(netD.state_dict(), '{}\u002FnetD_epoch_{}.pth'.format(outf, epoch + 1))\n\nwriter.close()\n",[45,552,553,557,562,566,571,576,581,586,590,595,600,605,610,614,619,624,629,634,639,644,649,655,661,666,672,678,683,689,695,700,706,711,717,722,728,734,740,745,751,757,763,768,774,780,785,791,797,802,808,814,819,825,830,836,842,848,854,860,866,872,878,883,889,895,901,906,912,918,924,930,935,941,947,953,958,964,970,976,982,987],{"__ignoreMap":141},[178,554,555],{"class":180,"line":181},[178,556,285],{},[178,558,559],{"class":180,"line":288},[178,560,561],{},"from datetime import datetime\n",[178,563,564],{"class":180,"line":294},[178,565,298],{"emptyLinePlaceholder":297},[178,567,568],{"class":180,"line":301},[178,569,570],{},"# log用フォルダを毎回生成\n",[178,572,573],{"class":180,"line":307},[178,574,575],{},"now = datetime.now()\n",[178,577,578],{"class":180,"line":313},[178,579,580],{},"log_path = \".\u002Flog\u002F\" + now.strftime(\"%Y%m%d-%H%M%S\") + \"\u002F\"\n",[178,582,583],{"class":180,"line":318},[178,584,585],{},"print(log_path)\n",[178,587,588],{"class":180,"line":324},[178,589,298],{"emptyLinePlaceholder":297},[178,591,592],{"class":180,"line":330},[178,593,594],{},"img_list = []\n",[178,596,597],{"class":180,"line":335},[178,598,599],{},"G_losses = np.array([])\n",[178,601,602],{"class":180,"line":341},[178,603,604],{},"D_losses = np.array([])\n",[178,606,607],{"class":180,"line":347},[178,608,609],{},"iters = 0\n",[178,611,612],{"class":180,"line":352},[178,613,298],{"emptyLinePlaceholder":297},[178,615,616],{"class":180,"line":358},[178,617,618],{},"# 学習のループ\n",[178,620,621],{"class":180,"line":364},[178,622,623],{},"for epoch in range(n_epoch):\n",[178,625,626],{"class":180,"line":370},[178,627,628],{},"    for itr, data in enumerate(dataloader):\n",[178,630,631],{"class":180,"line":376},[178,632,633],{},"        real_image = data[0].to(device)     # 元画像\n",[178,635,636],{"class":180,"line":381},[178,637,638],{},"        sample_size = real_image.size(0)    # 画像枚数\n",[178,640,641],{"class":180,"line":387},[178,642,643],{},"        noise = torch.randn(sample_size, nz, 1, 1, device=device)   # 正規分布からノイズを生成\n",[178,645,647],{"class":180,"line":646},20,[178,648,298],{"emptyLinePlaceholder":297},[178,650,652],{"class":180,"line":651},21,[178,653,654],{},"        real_target = torch.full((sample_size,), 1., device=device)\n",[178,656,658],{"class":180,"line":657},22,[178,659,660],{},"        fake_target = torch.full((sample_size,), 0., device=device)\n",[178,662,664],{"class":180,"line":663},23,[178,665,298],{"emptyLinePlaceholder":297},[178,667,669],{"class":180,"line":668},24,[178,670,671],{},"        # 識別器Dの更新\n",[178,673,675],{"class":180,"line":674},25,[178,676,677],{},"        netD.zero_grad()    # 勾配の初期化\n",[178,679,681],{"class":180,"line":680},26,[178,682,298],{"emptyLinePlaceholder":297},[178,684,686],{"class":180,"line":685},27,[178,687,688],{},"        output = netD(real_image)   # 識別器Dで元画像に対する識別信号を出力\n",[178,690,692],{"class":180,"line":691},28,[178,693,694],{},"        errD_real = criterion(output, real_target)  # 元画像に対する識別信号の損失値\n",[178,696,698],{"class":180,"line":697},29,[178,699,298],{"emptyLinePlaceholder":297},[178,701,703],{"class":180,"line":702},30,[178,704,705],{},"        D_x = output.mean().item()\n",[178,707,709],{"class":180,"line":708},31,[178,710,298],{"emptyLinePlaceholder":297},[178,712,714],{"class":180,"line":713},32,[178,715,716],{},"        fake_image = netG(noise)    # 生成器Gでノイズから贋作画像を生成\n",[178,718,720],{"class":180,"line":719},33,[178,721,298],{"emptyLinePlaceholder":297},[178,723,725],{"class":180,"line":724},34,[178,726,727],{},"        output = netD(fake_image.detach())  # 識別器Dで元画像に対する識別信号を出力\n",[178,729,731],{"class":180,"line":730},35,[178,732,733],{},"        errD_fake = criterion(output, fake_target)  # 贋作画像に対する識別信号の損失値\n",[178,735,737],{"class":180,"line":736},36,[178,738,739],{},"        D_G_z1 = output.mean().item()\n",[178,741,743],{"class":180,"line":742},37,[178,744,298],{"emptyLinePlaceholder":297},[178,746,748],{"class":180,"line":747},38,[178,749,750],{},"        errD = errD_real + errD_fake    # 識別器Dの全体の損失\n",[178,752,754],{"class":180,"line":753},39,[178,755,756],{},"        errD.backward()    # 誤差逆伝播\n",[178,758,760],{"class":180,"line":759},40,[178,761,762],{},"        optimizerD.step()   # Dのパラメーターを更新\n",[178,764,766],{"class":180,"line":765},41,[178,767,298],{"emptyLinePlaceholder":297},[178,769,771],{"class":180,"line":770},42,[178,772,773],{},"        # 生成器Gの更新\n",[178,775,777],{"class":180,"line":776},43,[178,778,779],{},"        netG.zero_grad()    # 勾配の初期化\n",[178,781,783],{"class":180,"line":782},44,[178,784,298],{"emptyLinePlaceholder":297},[178,786,788],{"class":180,"line":787},45,[178,789,790],{},"        output = netD(fake_image)   # 更新した識別器Dで改めて贋作画像に対する識別信号を出力\n",[178,792,794],{"class":180,"line":793},46,[178,795,796],{},"        errG = criterion(output, real_target)\n",[178,798,800],{"class":180,"line":799},47,[178,801,298],{"emptyLinePlaceholder":297},[178,803,805],{"class":180,"line":804},48,[178,806,807],{},"        errG.backward()     # 誤差逆伝播\n",[178,809,811],{"class":180,"line":810},49,[178,812,813],{},"        D_G_z2 = output.mean().item()\n",[178,815,817],{"class":180,"line":816},50,[178,818,298],{"emptyLinePlaceholder":297},[178,820,822],{"class":180,"line":821},51,[178,823,824],{},"        optimizerG.step()   # Gのパラメータを更新\n",[178,826,828],{"class":180,"line":827},52,[178,829,298],{"emptyLinePlaceholder":297},[178,831,833],{"class":180,"line":832},53,[178,834,835],{},"        if itr % display_interval == 0:\n",[178,837,839],{"class":180,"line":838},54,[178,840,841],{},"            print('[{}\u002F{}][{}\u002F{}] Loss_D: {:.3f} Loss_G: {:.3f} D(x): {:.3f} D(G(z)): {:.3f}\u002F{:.3f}'\n",[178,843,845],{"class":180,"line":844},55,[178,846,847],{},"                  .format(epoch + 1, n_epoch,\n",[178,849,851],{"class":180,"line":850},56,[178,852,853],{},"                          itr + 1, len(dataloader),\n",[178,855,857],{"class":180,"line":856},57,[178,858,859],{},"                          errD.item(), errG.item(), D_x, D_G_z1, D_G_z2))\n",[178,861,863],{"class":180,"line":862},58,[178,864,865],{},"        if epoch == 0 and itr == 0:     # 初回に元画像を保存する\n",[178,867,869],{"class":180,"line":868},59,[178,870,871],{},"            vutils.save_image(real_image, '{}\u002Freal_samples.png'.format(outf),\n",[178,873,875],{"class":180,"line":874},60,[178,876,877],{},"                              normalize=True, nrow=10)\n",[178,879,881],{"class":180,"line":880},61,[178,882,298],{"emptyLinePlaceholder":297},[178,884,886],{"class":180,"line":885},62,[178,887,888],{},"        # Save Losses for plotting later\n",[178,890,892],{"class":180,"line":891},63,[178,893,894],{},"        G_losses = np.append(G_losses,errG.item())\n",[178,896,898],{"class":180,"line":897},64,[178,899,900],{},"        D_losses = np.append(D_losses,errD.item())\n",[178,902,904],{"class":180,"line":903},65,[178,905,298],{"emptyLinePlaceholder":297},[178,907,909],{"class":180,"line":908},66,[178,910,911],{},"        # Tensorboard用のデータ\n",[178,913,915],{"class":180,"line":914},67,[178,916,917],{},"        writer.add_scalar(\"G_loss\",errG.item(),iters)\n",[178,919,921],{"class":180,"line":920},68,[178,922,923],{},"        writer.add_scalar(\"D_loss\",errD.item(),iters)\n",[178,925,927],{"class":180,"line":926},69,[178,928,929],{},"        iters += 1\n",[178,931,933],{"class":180,"line":932},70,[178,934,298],{"emptyLinePlaceholder":297},[178,936,938],{"class":180,"line":937},71,[178,939,940],{},"    if epoch % 10 == 0:\n",[178,942,944],{"class":180,"line":943},72,[178,945,946],{},"        fake_image = netG(fixed_noise)\n",[178,948,950],{"class":180,"line":949},73,[178,951,952],{},"        writer.add_image(\"fake_image\",vutils.make_grid(fake_image), epoch)\n",[178,954,956],{"class":180,"line":955},74,[178,957,298],{"emptyLinePlaceholder":297},[178,959,961],{"class":180,"line":960},75,[178,962,963],{},"    # モデルの保存\n",[178,965,967],{"class":180,"line":966},76,[178,968,969],{},"    if (epoch + 1) % 50 == 0:   # 50エポックごとにモデルを保存する\n",[178,971,973],{"class":180,"line":972},77,[178,974,975],{},"        torch.save(netG.state_dict(), '{}\u002FnetG_epoch_{}.pth'.format(outf, epoch + 1))\n",[178,977,979],{"class":180,"line":978},78,[178,980,981],{},"        torch.save(netD.state_dict(), '{}\u002FnetD_epoch_{}.pth'.format(outf, epoch + 1))\n",[178,983,985],{"class":180,"line":984},79,[178,986,298],{"emptyLinePlaceholder":297},[178,988,990],{"class":180,"line":989},80,[178,991,390],{},[14,993,994],{},"500 エポック回した結果が以下になります。",[446,996],{"alt":997,"img-src":998},"result","\u002Fimg\u002Ftensorboard\u002Fresult.png",[14,1000,1001,1002,1004],{},"うーんな画像が生成されてます。",[252,1003],{},"\nもうちょっと回せばいいのがでそうかなという感じです。",[14,1006,1007,1008,1010,1011,1013],{},"微妙な結果になった原因はおそらく画像枚数を 1000 枚で学習させたからだと思います。",[252,1009],{},"\nマシンスペック的に celeba の画像全部は無理でした。",[252,1012],{},"\n他のパラメータは特に変えてないです。",[14,1015,1016],{},"以下は生成器と識別器のロスになります。",[446,1018],{"alt":1019,"img-src":1020},"result_loss","\u002Fimg\u002Ftensorboard\u002Fresult_loss.png",[14,1022,1023],{},"学習は順調に進んでるように見えます。",[14,1025,1026],{},"検証結果の紹介は以上です！",[10,1028,1030],{"id":1029},"ハマったところ-1","ハマったところ 1",[14,1032,1033],{},"まずは環境構築の部分で以下のエラーに悩みました。",[446,1035],{"alt":1036,"img-src":1037},"error","\u002Fimg\u002Ftensorboard\u002Ferror.png",[14,1039,1040,1041,1044],{},"この",[24,1042,1043],{},"thinc","というやつのエラーがどうやっても回避できなかったです。",[14,1046,1047],{},"結局、このエラーを放置しても作業に支障はなかったのですが、なんかモヤっとします。",[14,1049,1050],{},"今後、対策をちゃんと調べようと思います。",[10,1052,1054],{"id":1053},"ハマったところ-2","ハマったところ 2",[14,1056,1057],{},"次に、作業は jupyter notebook で行っていたのですが、TensorBoard がリセットされない問題が発生しました。",[14,1059,1060,1061,1063],{},"どういう現象かというと、お試しで作成したデータを削除して、",[252,1062],{},"\n新しく生成されたデータのみを TensorBoard で可視化しようとしたところ、\n古いデータがグラフに残ったまま次のデータが同一のグラフに描画されてしまいました。",[14,1065,1066],{},"元の tfevent ファイルを削除してもどうにもならなかったです。",[14,1068,1069],{},"とりあえずグラフが重なってしまうのは評価の上で論外だったので、一応対策しました。\n対策は 3 つほど見つけました。",[1071,1072,1073,1076,1083],"ol",{},[89,1074,1075],{},"tfevent は常に新しいディレクトリに保存する",[89,1077,1078,1079,1082],{},"可視化部分を含むコードを変更した場合は、jupyter notebook の kernel から",[45,1080,1081],{},"Restart & Clear Output","を行う",[89,1084,1085,1088],{},[45,1086,1087],{},"tf.reset_default_graph()","を使う",[14,1090,1091],{},"手っ取り早く確実に効果があるのは一つ目です。",[14,1093,1094],{},"TensorBoard は log ディレクトリに指定した配下ではディレクトリごとに表示を分けることができます。",[14,1096,1097],{},"なので、実行のたびに新しくディレクトリを作成し、tfevent ファイルはその下に保存されるようにすると、グラフが重なることなく表示できます。",[14,1099,1100],{},"2 と 3 番目は試したのですが、効果がみられなかったです。（使い方が違った可能性あり）",[10,1102,1103],{"id":1103},"感想",[14,1105,1106,1107,1109],{},"TensorBoard 便利すぎて感動です。",[252,1108],{},"\n導入、学習コストも低く、でてくる出力の見た目も申し分ないです。",[14,1111,1112,1113,1115],{},"またリアルタイム可視化も便利でした。",[252,1114],{},"\nリアルタイムに生成画像やロスを見れるので GAN の学習にぴったりでした。",[14,1117,1118],{},"jupyter notebook から直接開けるのも使いやすくてポイント高いです。",[10,1120,1121],{"id":1121},"参考",[86,1123,1124,1131],{},[89,1125,1126],{},[160,1127,1130],{"href":1128,"rel":1129},"https:\u002F\u002Famzn.to\u002F2O6B3Tt",[164],"PyTorch ニューラルネットワーク実装ハンドブック (Python ライブラリ定番セレクション)",[89,1132,1133],{},[160,1134,1137],{"href":1135,"rel":1136},"https:\u002F\u002Fqiita.com\u002Fnj_ryoo0\u002Fitems\u002Ff3aac1c0e92b3295c101",[164],"PyTorch の TensorBoard サポートを試してみる",[1139,1140,1141],"style",{},"html pre.shiki code .svObZ, html code.shiki .svObZ{--shiki-default:#B392F0}html pre.shiki code .sU2Wk, html code.shiki .sU2Wk{--shiki-default:#9ECBFF}html pre.shiki code .sDLfK, html code.shiki .sDLfK{--shiki-default:#79B8FF}html .default .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}html .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}",{"title":141,"searchDepth":288,"depth":288,"links":1143},[1144,1145,1146,1147,1148,1149,1150,1151,1152],{"id":12,"depth":288,"text":12},{"id":33,"depth":288,"text":34},{"id":59,"depth":288,"text":60},{"id":109,"depth":288,"text":110},{"id":509,"depth":288,"text":510},{"id":1029,"depth":288,"text":1030},{"id":1053,"depth":288,"text":1054},{"id":1103,"depth":288,"text":1103},{"id":1121,"depth":288,"text":1121},"2019-09-17","Pytorchでディープラーニングの簡単な可視化したくないですか？本記事では、PytorchでTensorBoardを使った簡単な可視化方法とDockerを用いた導入方法を紹介します！実際に、TensorBoardをDCGANのPytorch実装で試してみたのでぜひ見てください！",false,"md",{},"\u002Fcontents\u002Ftensorboard",{"title":5,"description":1154},"contents\u002Ftensorboard",[1162,1163],"PyTorch","機械学習","\u002Fimg\u002Ftwitter-card.png","my0ADjT2WFRpyh0d9RaTpvciMIW7Rb8OeiGg5o2Zw74",[1167,1171],{"title":1168,"path":1169,"stem":1170,"children":-1},"Tailwind Nextjs Starter Blog + Netlifyでブログ構築","\u002Fcontents\u002Ftailwondblog-nextjs","contents\u002Ftailwondblog-nextjs",{"title":1172,"path":1173,"stem":1174,"children":-1},"TTHE: Test-Time Harness Evolution","\u002Fcontents\u002Ftest-time-harness-evolution","contents\u002Ftest-time-harness-evolution",1784106046309]