Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Dynamic VM Scheduling in OpenStack
Search
Kazuhiko Yamashita
November 25, 2022
Programming
1
1.1k
Dynamic VM Scheduling in OpenStack
CNDT2022で話した内容です。
Kazuhiko Yamashita
November 25, 2022
Tweet
Share
More Decks by Kazuhiko Yamashita
See All by Kazuhiko Yamashita
ペパボOpenTelemetry革命
pyama86
2
1.1k
Site Reliability Engineering for GMO
pyama86
9
1.1k
PHPアプリケーションのスケーラビリティと 信頼性を革新する nginx+ngx_mrubyとGoの融合
pyama86
2
290
ペパボOpenTelemetry革命
pyama86
0
720
ChatGPTの活用を体現し、 組織に浸透する技術
pyama86
1
920
リリースフラグと プログレッシブデリバリーを活用した 安全なWEBアプリケーションデプロイ
pyama86
2
170
PHPerのためのOpenTelemetry入門
pyama86
1
230
生成AIで仕事を もっとおもしろくする事例と詳解
pyama86
0
150
Goで実装された高速な 仮想待合室サーバの実装と詳解
pyama86
15
7.1k
Other Decks in Programming
See All in Programming
TypeScriptで使いやすいOpenAPIの書き方
yukimochi_dwango
1
890
『Railsオワコン』と言われる時代に、なぜブルーモ証券はRailsを選ぶのか
free_world21
2
480
Deep Dive into React Stream/Serialize
mugi_uno
4
860
GoでParserを書く
karupanerura
3
370
Escolhendo (ou não) o melhor ORM para o seu projeto
andreiacsilva
1
160
TypeScriptコードの漸進的改善 / Progressive Improvement of TypeScript Code
medley
1
440
The Final Frontier of Web Development: React Server Components vs Jakarta EE
ivargrimstad
0
110
Implementing Design Systems in Swift
seyfoyun
2
530
GitLab CI/CD で C#/WPFアプリケーションのテストとインストーラーのビルド・デプロイを自動化する
hacarus
0
620
Ruby on Fails - effective error handling with Rails conventions
talyssonoc
0
300
otelcol receiver 自作RTA / Pepabo Tech Conference #22 春のSREまつり
arthur1
0
970
Long journey of Ruby standard library RubyKaigi 2024
andpad
2
230
Featured
See All Featured
Agile that works and the tools we love
rasmusluckow
325
20k
What’s in a name? Adding method to the madness
productmarketing
PRO
17
2.7k
Mobile First: as difficult as doing things right
swwweet
217
8.6k
ピンチをチャンスに:未来をつくるプロダクトロードマップ #pmconf2020
aki_iinuma
84
45k
Into the Great Unknown - MozCon
thekraken
15
1.1k
Debugging Ruby Performance
tmm1
70
11k
Automating Front-end Workflow
addyosmani
1357
200k
GraphQLの誤解/rethinking-graphql
sonatard
56
9.3k
Rebuilding a faster, lazier Slack
samanthasiow
74
8.3k
Dealing with People You Can't Stand - Big Design 2015
cassininazir
358
22k
Code Review Best Practice
trishagee
56
15k
Keith and Marios Guide to Fast Websites
keithpitt
408
22k
Transcript
Dynamic VM Scheduling in OpenStack ʙ͜ͷલɺθΫγΟಡΜ͚ͩͲɺRFCΑΓ͔ͬͨ͠ฤʙ
ࢁԼ!QZBNB (.0ϖύϘٕज़ج൫νʔϜ γχΞɾϓϦϯγύϧ UFOTOBQPODPN QZBNBGVO TUOTKQ
45/4 -JOVY/444FSWFS TUOTKQ
45/4
None
CNDT ٱ͠ͿΓʹΦϑϥΠϯͰ ࠷ߴ!!1 Ͱ͜Εը͚ͩͲͶ
ϖύϘͷΠϯϑϥߏ CPU StealʹىҼͨ͠ɺྡਓ εʔύʔˑϥΠϒϚΠάϨʔγϣϯ ·ͱΊͱɺ݁ࠗʹ͓͚Δେࣄͳ͜ͱ42બ ࠓ͢͜ͱ
ϖύϘͷΠϯϑϥߏ 0QFO4UBDL #BSFNFUBM "84 IFSPLV ($1 ཧαʔό7.ن ϗεςΟϯάܥͷαʔϏεɺ֤छج൫αʔϏεͰར༻ &,4
3%4 &MBTUJDYYYܥͷϚωʔδυαʔϏεΛத৺ʹར༻ (,& #JH2VFSZΛத৺ʹ"84ͷ%3ɺσʔλΤϯδχΞϦϯάͷج൫ͱͯ͠ར༻
0QFO4UBDL 0QFO4UBDLͰཧ͢Δ7.ͷ্ʹ ࣗࣾ։ൃͷΤϯδϯΛར༻ͯ͠ ,VCFSOFUFTΛσϓϩΠ 0QFO4UBDL #BSFNFUBM #BSFNFUBM #BSFNFUBM 7.
7. 7. ,VCFSOFUFT $POUBJOFS $POUBJOFS $POUBJOFS ,VCFSOFUFT
CPU StealʹىҼͨ͠ɺ ྡਓ
ॠؒɺ৺ɺॏͶͯ ຊ࣌ؒͰαʔϏεͬͯΔ͔Β ϐʔΫλΠϜ͕ॏෳ 9:00 12:00 20:00 ϐʔΫλΠϜ ϐʔΫλΠϜ ϐʔΫλΠϜ ϐʔΫλΠϜ
ϐʔΫλΠϜ ϐʔΫλΠϜ
CPU Steal Ϧιʔε͕ୣ͍߹͍ʹͳΓɺCPU Steal͕ൃੜ
CPU Stealͷ֓೦ Hypervisor VM1 CPU͍͍ͨʂ 10ඵ͋͛Δ VM2 CPU͍͍ͨʂ 10ඵ͋͛Δ
VMڥʹ͓͚ΔCPU Stealͷ֓೦ Hypervisor VM1 CPU͍͍ͨʂ 10ඵ͋͛Δ VM2 CPU͍͍ͨʂ 10ඵ͋͛Δ ࣮ࡍ8ඵ͔͋͛͠ΒΕͳ͔ͬͨ
ࠩͷ2ඵ͕CPU Steal
CPU Steal͕ى͖Δͱʁ αʔόॲཧೳྗͷԼ αʔόϝτϦΫεͷѱԽ HTTPͷϨΠςϯγͷѱԽ ଓͷ૿Ճ ϩʔυΞϕϨʔδͷ্ঢ
εʔύʔˑϥΠϒϚΠάϨʔγϣϯ
ϥΠϒϚΠάϨʔγϣϯ Hypervisor VM1 VM2 10ඵ CPU 2ඵ CPU Hypervisor VM1
10ඵ CPU VMΛఀࢭ͢Δ͜ͱͳ͘HypervisorؒΛҠಈ
ϥΠϒϚΠάϨʔγϣϯ Hypervisor Hypervisor VM1 Disk Memory VM1 Disk STEP 1
σΟεΫΛಉظ ※ڞ༗ετϨʔδΛར༻͍ͯ͠ͳ͍߹
ϥΠϒϚΠάϨʔγϣϯ Hypervisor Hypervisor VM1 Disk Memory VM1 Disk STEP 2
ϝϞϦΛಉظͯ͠੩ࢭΛͭ ※ڞ༗ετϨʔδΛར༻͍ͯ͠ͳ͍߹ Memory
OpenStackͷϥΠϒϚΠάϨʔγϣϯ https://docs.openstack.org/nova/pike/admin/con fi guring-migrations.html
ϥΠϒϚΠάϨʔγϣϯ Hypervisor Hypervisor VM1 Disk Memory VM1 Disk Memory STEP
3 ੩ࢭ͕औΕͨΒ৽͍͠ΠϯελϯεΛ ༗ޮʹͯ͠ɺωοτϫʔΫܦ࿏ΛΓସ͑Δ ͓લ͕ࠓ͔ΒVM1ͩ
εʔύʔˑϥΠϒϚΠάϨʔγϣϯ
εʔύʔˑϥΠϒϚΠάϨʔγϣϯ 1. VMʹ͓͍ͯStealͷൃੜΛݕ͍ͨ͠ 2.Stealͷൃੜ͍ͯ͠ͳ͍HypervisorʹεέδϡʔϦϯά͍ͨ͠ 3.μϯλΠϜΛൃੜͤͨ͘͞ͳ͍ 4.ࣗಈͰ৸ͯΔؒʹͯ͢ऴΘͬͯ΄͍͠ ͱ͍͏ͷΛɺRubyͰ࣮ͨ͠
VMͷStealൃੜݕ Load5͕͖͍͠Ҏ্͔ͭɺSteal͕ൃੜ͍ͯ͠Δ͜ͱ Steal͕ൃੜͯ͠ɺॲཧ͕Ԇ͍ͯ͠ͳ͍ͳΒͳ͍
εέδϡʔϦϯά HypervisorͷϝτϦΫεͰSteal͕͖͍͠ҎԼͰɺΑΓۭ͖͕͋Δ ಉ͡ϩʔϧͷVM͕ಉҰαʔόʹ͖͍͠Ҏ্ऩ༰͞Ε͍ͯͳ͍ ಉ͡AZͷαʔόͰ͋Δ
Availability Zone LB AZ-1 VM1 AZ-2 VM2 LB AZ-1 VM1
VM2 AZ-2 AZΛ·͍ͨͰεέδϡʔϦϯά͢Δͱ োੑ͕ഁ໓!!1
μϯλΠϜΛൃੜͤ͞ͳ͍ ΦʔέετϨʔλʔͰ͢Δ $ kubectl drain example.local --force --ignore-daemonsets —delete-emptydir-data #
after migrated $ kubectl uncordon example.local $ consul maint -enable # after migrated $ consul maint -disable Kubernetes Consul
kubectl drain node-1 pod-1 Scheduling Disabled node-2 pod-2 pod-1 nodeͷPodͷׂΛఀࢭͯ͠ɺ͍ग़͢
consul node-1 (10.0.0.1) nginx node-2 (10.0.0.2) nginx node-3
(10.0.0.3) nginx $ dig web.service.consul a 10.0.0.1 10.0.0.2 10.0.0.3 ϔϧενΣοΫͳͲΛར༻ͯ͠ɺαʔϏεσΟεΧόϦ͕Ͱ͖Δ DNSͷଞʹɺConsul TemplateΛར༻ͨ͠ ߏϑΝΠϧͷϨϯμϦϯάͳͲͰ׆༻͍ͯ͠Δ
consul maint -enable node-1 (10.0.0.1) nginx node-2 (10.0.0.2)
nginx node-3 (10.0.0.3) nginx $ dig web.service.consul a 10.0.0.1 10.0.0.2 Ϋϥελ͔ΒΓ͠ consul maint -enable
ॲཧܥ·ͱΊ Stealͷൃੜ͔ͭ ϩʔυΞϕϨʔδͷ্ঢ ࠷దͳ؋ͷબఆ Ϋϥελ͔ΒͷΓ͠ ϥΠϒϚΠάϨʔγϣϯ ΫϥελͷΈࠐΈ ࣮ߦڥͱͯ͠OpenStackͷ্Ͱಈ͘ Kubernetesͷதͷpodͱͯ͠ɺ Reconciliation
loopͬΆ͘ಈ͍͍ͯΔ
ຊʹ͋ͬͨා͍
ʮεʔύʔˑϥΠϒϚΠάϨʔγϣϯ ಈ͍ͯΔΜͰ͚͢ͲɺϨΠςϯγ ߴ͍ΜͰ͢ΑͶʯ
ϥΠϒϚΠάϨʔγϣϯ͕ଳҬΛ৯͍Δ ͓ன͔Βഁ໓!!1
ϥΠϒϚΠάϨʔγϣϯ(࠶ܝ) Hypervisor Hypervisor VM1 Disk Memory VM1 Disk STEP 1
σΟεΫΛಉظ ※ڞ༗ετϨʔδΛར༻͍ͯ͠ͳ͍߹ ʊਓਓਓਓʊ ʼɹ500Gɹʻ ʉY^Y^Y^Y^ʉ
None
None
·ͱΊ ྡਓΛࣗͰεέδϡʔϥʔॻ͘͜ͱͰղܾͨ͠ OSSʹ͢ΔͱdeschedulerͷΑ͏ͳநʹͳΔͷͰɺ ࠓճެ։͠ͳ͔ͬͨ ϥΠϒϚΠάϨʔγϣϯҙ֎ͱଳҬ৯͏ͷͰɺωοτϫʔΫΛͯ͠ ϨʔτϦϛοτೖΕͨ΄͏͕͍͍
ଟɺձʹ͍ΔͷͰɺ టʹͳΓ·͠ΐ͏ʂʂʂ
None
͓͠·͍ ࠷৽ͷ࠾༻ใΛνΣοΫˠ !QC@SFDSVJU