æ代ã¯ã¯ã©ãŠãããããã財åžãâŠâŠ
AWSãAzureãGCPãOCIãªã©å€§æITã¯ã¯ã©ãŠããµãŒãã¹ãæããç±å¿ã«æšé²ããŠããŸããå人ã§ãã¯ã©ãŠããåŠç¿ããããããã«ãç¡ææ ãã¯ãŒãã³ãæ¡å ãããã³ããŒãå¢ããŠããŸããã
ã¯ã©ãŠãã¯PCãLANãªã©ã®åææè³ãäžèŠã§ã³ã¹ãçã«ãæ·å± ã¯äœããé«æ§èœPCãäžåãåŠçèœåã®ä»®æ³ãã·ã³(VM)ã€ã³ã¹ã¿ã³ã¹ãæäŸãããŠããŸãã
ããããåžžæ皌åãããšã©ã³ãã³ã°ã³ã¹ãã1å°ãããæ1äžåãè¶ ããŠããŸããç§ã¯90幎代ããã¬ã³ã¿ã«ãµãŒããŒã§åœãµã€ããéå¶ããŠããŸããããæ®éã®ãµã©ãªãŒãã³ãšããŠèªè ¹ã§ãµãŒããŒè²»çšãè² æ ããã®ã¯æ 1äžåããããéçã§ããã
ã¯ã©ãŠããã³ã¹ãçã«æå©ãªã®ã¯ãå¿ èŠãªãã®ãå¿ èŠãªãšãã ãã調éããããã§ããå ·äœçã«ã¯åœWebãµã€ãã®ããã«æå°éã®ãªãœãŒã¹ã ãåžžæ皌åããå Žåããããã¯æ©æ¢°åŠç¿ãªã©ã®ããã«é«æ§èœGPUãªã©èŽ æ²¢ãªãªãœãŒã¹ãäžå®æéã ãçšããå Žåãªã©ã«éãããŸããæè¡åãé«ãæ¹ãªããããŒã¿åŠçã¿ã¹ã¯ã«ãã課éãããªããµãŒããŒã¬ã¹ã¢ãŒããã¯ãã£ã§ã·ã¹ãã å šäœãæ§ç¯ããããšãéžæè¢ãšãªãåŸãŸãã
ãããã«ãããã䜿ãæŸé¡ã¯ã¯ã©ãŠãã§ã¯å®çŸåºæ¥ãªã(çŸå®çãªè²»çšã§ã¯)ããšããããšã§ããäŸå€ãšããŠãåžžæ皌åããåºæ¥ãŸãããæ1,000åã§GPUã€ã³ã¹ã¿ã³ã¹ã䜿ãæŸé¡ãªColab ProããããããŒã¿åºç€ãšããŠå人ãå¥çŽåºæ¥ãã¯ã©ãŠãã®éçããšã®èªèã§ãã
Excelã§åºæ¥ãããŒã¿åæã«ããŒã¿åºç€ã¯èŠããªã
ããã§ãã£ããåç¹ã«æ»ãããå人ãäœã®ããã«ããŒã¿åºç€ãæ§ç¯ããã®ãïŒããšããããŒããèå¯ããŸãã
åŠæ ¡ã®èª²é¡ãä»äºã§ãåããŠããŒã¿åæã«åãçµãã ãšãã®ããšãæãåºããŠã¿ãŸããããã»ãšãã©ã®äººã¯Excelã ã£ããšæããŸããæ©æ¢°åŠç¿çéã§ã¯ããªãããããªãPythonããšããã¹ããŒãªãŒãå€ãã§ããããããPythonãRã¯Excelããå§åçã«æ·å± ãé«ãã§ããããæ¬æ¥ã¯ãExcelã§ã¯ç¡çã ãããPythonãRã«åãçµãã®ãçããšãããã®ã§ããããå ·äœçã«ã¯
- Excelã«ãªãæ©èœã䜿ãããã®ã§PythonãRãªã©ã®èšèªã䜿ã
- Excelã§ã¯åŠçåºæ¥ãªãã»ã©å·šå€§ãªããŒã¿ãæäœãããã®ã§SQLããŒã¿ããŒã¹ã䜿ã
- ã®ã¬ãã€ãããã©ãã€ãåäœã®ããŒã¿ããŒã¹ã管çãããã®ã§ããŒã¿åºç€ãæ§ç¯ãã
ãšãã£ãåæ©ã§ããã¯ãã§ããããã£ãŠããªãé«åºŠã§èŽ æ²¢ã§ãããããŸãExcelãGoogleã¹ãã¬ããã·ãŒããè©ŠããŠã¿ãŠãããŒã¿åæã®ç®çãæ¬åœã«ExcelãGoogleã¹ãã¬ããã·ãŒãã§è¶³ããªãã®ã確èªããããšããå§ãããŸããç¹ã«Googleã¹ãã¬ããã·ãŒãã¯PCçã®Excelã§ã¯ããã«ãããŠã§ãããã®ããŒã¿ååŸãç¡æã§åºæ¥ãŸãã®ã§ãéãããããŒã¿ãããã°ããŒã¿ã§ãªããã°æåã«æ€èšããã¹ãéžæè¢ã ãšæããŸãã
å人ããŒã¿åºç€ãäœãã®ã¯ãããã°ããŒã¿ãèŽ æ²¢ã«äœ¿ãããããã
ãããŸã§æ¥ããšãå人ã§ããŒã¿åºç€ãäœãã®ã¯ãããã°ããŒã¿ãèŽ æ²¢ã«äœ¿ããããããã ãšããããšãèŠããŠããŸããæ¬æ¥ãªãäŒæ¥ã倧åŠã§ããæ±ããªããããªå·šå€§ãªããŒã¿ãæ¢ããŠå人ã§èç©ã»æŽ»çšãããã®ãåæ©ã§ããããç³æ²¹çã§ããªããã°ã¯ã©ãŠãã§ã¯ç¡çãšããããšã«ãªããŸãã
ãªã³ãã¬ãšèšã£ãŠããPC1å°ã§åºæ¥ãã¿ã¹ã¯ãªããããŒã¿åºç€ããªã©ãšããä»°ã ãããã®ãå人ã§äœãå¿ èŠã¯ãããŸãããããã¯ã¢ããã®èŠ³ç¹ã§ãGoogleãã©ã€ãã§ã¯100GBã幎é¡2,500åã§ããããå°æ¥çã«1TBãè¶ ãããããªããŒã¿ã管çããããããªäººããããŒã¿åºç€ããæ€èšãã䟡å€ããã人ããšããããšã«ãªããŸãã
å人ããŒã¿åºç€ã§æ³å®ããã¢ãŒããã¯ãã£ïŒãPCã¯ã©ã¹ã¿ã
å人ã§ãã©ãã€ãåäœã®ããŒã¿ã倧éã«èç©ããŠå©æŽ»çšããã«ã¯ããããã«ã¯ã©ãŠããPC1å°ã§ã¯ç¡çããããŸããããã§è€æ°å°ã®PCãé£æºãããPCã¯ã©ã¹ã¿ããèããŠã¿ãŸãã
å人çšPCã¯æè¿ã§ã¯ããŒãPCãäž»æµã§ãããã¿ã¯ãŒåã®ãããªå€§åPCãè€æ°å°èªå® ã«çœ®ãã®ã¯çŸå®çã§ã¯ãããŸããããããã³ãŒã«ã»ã³ã¿ãŒãåä»çšã«è²©å£²ãããŠããè¶ å°åPCãªãã5å°ãããèªå® ã«çœ®ããŠãå Žæã¯ãã»ã©åããŸãããç§ã®èªå® ã§ã¯ãå®éã«è¶ å°åPCã10å°ã»ã©çšŒåããŠããŸãã
1å°ãããã®æ¶è²»é»åã¯30WååŸã§ãé»æ°æéã¯1å°ãããæé¡æ°çŸåçšåºŠã§ããé»æ°æéã ãã§èŠãã°ããªã³ãã¬ããŒã¿åºç€ã®ã©ã³ãã³ã°ã³ã¹ãã¯ã¯ã©ãŠãã®100åã®1ããšããããšã«ãªããŸãã
ããŒã¿åºç€ãšããŠã®PCã¯ã©ã¹ã¿å©çšãæ³å®ããå ŽåãLANã®é床ãæ·±å»ãªããã«ããã¯ãšãªããŸããçŸç¶ã§ã¯ã³ã¹ããšã®å Œãåããã2.5GbEã§æ§ç¯ããã®ãçŸå®çã§ããå°ããå€æ®µã匵ããŸãã2.5GbE察å¿ã®ãããšUSBæ¥ç¶ã®LANã¢ããã¿ãŒãçšæããããšããå§ãããŸãã
OSã¯ãã¢ããããŒãæã®éä¿¡éã®å°ãªãããµãŒããŒç®¡çã®å®¹æããªã©ããUbuntu Linuxã䜿çšããŠããŸããçŸåšã¯Ubuntu 20.04 LTSã䜿çšããŠããŸããã2022幎4ææ«ã«ãªãªãŒã¹äºå®ã®Ubuntu 22.04 LTSã«å ¥ãæ¿ããäºå®ã§ããUbuntu 22.04 LTSã¯ãæè¿ã®AMD Ryzen APUã§ãçŽ ã®ããã©ç¶æ ã§åãåã®LTSãªãªãŒã¹ãšãªããã倧ãã«æåŸ ããŠããŸãããªããåŸã«ã玹ä»ãããåæ£ããŒã¿ããŒã¹ãã¯å€§åãJava VMäžã§åäœãããããäžå€PCãã³ãã«ã§å®äŸ¡ã«å ¥æåºæ¥ãWindows Professionalãªã©äžè¬ã®æ¹ã䜿ãæ £ããOSã§ãæ§ç¯éçšèªäœã¯å¯èœãšæãããŸãã
念ã®ããè£è¶³ããŠãããŸãããè¶ å°åPCãã¯ã©ã¹ã¿éçšããéã«ã¯ãã¢ãã¿ãŒãããŒããŒãã¯éªéãªã®ã§å€ããŠããŸããå¿ èŠã«å¿ããŠHDMI/DisplayPortã§æ¥ç¶åºæ¥ãã¢ãã€ã«ã¢ãã¿ãŒãããŒããŒããšã¿ããããããäžäœåããå ¥åè£ çœ®ãæ¥ç¶ããŠã¡ã³ããã³ã¹ãè¡ããŸããéšå±ãæããªãæ¹ã¯å ã®åçã®ããã«è¶ å°åPCãšåãµã€ãºã®USBãã¡ã³ãéããŠèšçœ®ããŠãããšå€å Žãå°ãå®å¿ã§ãã
SQL or NoSQL?
PCã¯ã©ã¹ã¿åãã®ããŒã¿ç®¡çã«ã¯å°çšã®ãœãããŠã§ã¢ãå¿ èŠã§ããã¹ã¿ã³ãã¢ãã³ã®ããŒã¿ããŒã¹ã¯å®æçãªããã¯ã¢ããã§ããŒã¿ãä¿å šããŸãããPCã¯ã©ã¹ã¿åãã®åæ£ããŒã¿ããŒã¹ã¯ããŒã¿æ¬äœãåå²ããŠè€æ°ã®ãã¬ããªã«ããšããŠãµãŒããŒã«åæ£æ ŒçŽãããã·ã£ãŒãã£ã³ã°ããšããä»çµã¿ã§åããŸãã
ãããŸã§åæ£ããŒã¿ããŒã¹ã«ã¯NoSQLã®ãElasticsearchãã䜿ã£ãŠããŸããããããElasticsearchã¯Kibanaãªã©å瀟ã®BIããŒã«ãšé£æºããŠçšããããšãåæãšããŠãããç¡åã®ç¯å²ã§ã¯æ±çšçãªJDBCã³ãã¯ã¿ãå©çšåºæ¥ãªãããšãªã©ããã次æã®ããŒã¿åºç€ã¯åæ£RDBã«æ¿ããããšæã£ãŠããŸãã
NoSQLã¯JSON&KeyValueã§ã®ããŒã¿æ ŒçŽãåæãšããŠããŸãããã°ããŒã¿ã®èç©ã«åããŠããæè»ãªããŒã¿æ§é ã«å¯Ÿå¿åºæ¥ããšããæå³ã§ã¯åªããŠããŸããããããããŒã¿åæã®éã¯ã»ãŒã»ãŒè¡šåœ¢åŒã«å å·¥ããŠçšããããšãããŒã¿æ ŒçŽæã«ããŒã¿åããã§ãã¯ããŠãããªããšåææã«ååŠçã§å°ãããšãå€ãããšãªã©ãããç§ã«ãšã£ãŠã¯SQL察å¿ã®åæ£RDBã§ããŒã¿ã管çããã®ãäžçªããšããçµè«ã«è³ããŸããã
ã¯ã©ãŠãã§ã¯ããã¿ã³ã²ãšã€ãã§äœ¿ããåæ£RDBãããããªã³ãã¬ã§ã¯âŠâŠ
åæ£RDBã¯åæ£ããŒã¿ããŒã¹ã®äžçš®ã§ãè€æ°ã®ãµãŒããŒã«ããŒã¿ãè€è£œããŠä¿åããã·ã£ãŒãã£ã³ã°ãšããæè¡ãçšããŸãããµãŒããŒã®æ°ãå¢ãããŠã¬ããªã«ãåæ§ç¯ããã°èªåçã«æ°ãããµãŒããŒã«ããŒã¿ãè€è£œãããŸããããµãŒããŒã®ãã¡1å°ãå£ããŠãä»ã®ãµãŒããŒã«ã¬ããªã«ãä¿åãããŠããã®ã§ããŒã¿ã倱ããã«æžã¿ãŸãã
å人ã§ãNASãªã©ã§RAIDãéçšããŠããæ¹ãå€ããšæããŸãããNASãã®ãã®ãå£ããŠããŸããšããŒã¿ã¯HDDãããµã«ããŒãžããªããã°ãªãããç¡äºãµã«ããŒãžåºæ¥ãããé次第ããšããã®ãçŸå®ã§ããããŒããŠã§ã¢é害æã«ãããŒã¿ãå®éçšåºæ¥ã圢ã§ä¿æããããã«ã¯ãPCã¯ã©ã¹ã¿ã§ããŒã¿ãã¬ããªã±ãŒã·ã§ã³ããŠåæ£éçšããã®ããªã³ãã¬ã§å¯äžã®æå¹ãªæ段ãšèããŠããŸãã
ãã®ãããªæè¡ã¯ã倧æã¯ã©ãŠãã§ã¯æ¢ã«åœããåã®ãã®ãšãªã£ãŠããŸããGoogleã®Cloud SpannerãããŒã¿åæã«ç¹åããBigQueryãAWSã®Amazon Redshiftãªã©ã¯ãŠãŒã¶ãŒãæèããªã圢ã§å·šå€§ãªåæ£ããŒã¿ããŒã¹ãæ§ç¯ããŠãããããã«ãããã¿ãã€ãçŽã®ããŒã¿ãæ±ãããšãããŠããŸããããŒã¿ã®ã¬ããªã±ãŒã·ã§ã³ããŠãŒã¶ãŒã«æèãããããšãªãè€æ°ã®ããŒã¿ã»ã³ã¿ãŒã«åæ£ããŠæ ŒçŽãããŠãããããããŒã¿ã®å ç¢æ§ãšããæå³ã§ã¯ã¯ã©ãŠããæ匷ã§ããããšã«çãã¯ãããŸããããã ããåœç¶ãªããå ç¢ãªåã ããé«ãã®ã§ãããŒã¿ä¿å šåœã§ã¯ã©ãŠãã«ããã°ããŒã¿ã眮ããã©ããã¯ãã財åžãšçžè«ãã«ãªããŸãã
ãããããªã³ãã¬çšã®åæ£RDBã«ã¯åŸæ¥ãããŒãããã®ããããŸããã§ããããªãŒãã³ãœãŒã¹ã®è£œåãããã€ãååšããŠã¯ããã®ã§ãããSQLã®å¯Ÿå¿ã匱ãã£ããJDBCãã©ã€ãã§æååããåºããªã©æ±çšçãªããŒã¿åºç€ãšããŠã¯å³ãããã®ããããŸãããéçºå ãã¢ã¡ãªã«ã§ã¯ãªãäŒç€ŸãšããŠã®äºæ¥ç¶ç¶æ§ãæªç¥æ°ããšããå°è±¡ã
åæ£RDBã®æ±ºå®æãšãªãã?PostgreSQLæ¡åŒµãCitusã
ãšãããæè¿ã«ãªã£ãŠããã€ã¯ããœããããAzure Database for PostgreSQL â Hyperscale (Citus) ãšããã¯ã©ãŠããµãŒãã¹ãåºãŠããŸãããããèªäœã¯ä»ç€Ÿã¯ã©ãŠãã®åæ£RDBãšå€§å·®ç¡ãã®ã§ããã
- å®çªã®OSS RDBã§ããPostgreSQLãæ¡åŒµãã圢ã§åæ£RDBãå®çŸããŠãã
- PostgreSQLã®æ¡åŒµã§ãããããBIãªã©åçš®ããŒã«ãçšæããŠããPostgreSQLçšã®JDBC/ODBCãã©ã€ãããã®ãŸãŸå©çšã§ãã(ããç¥ããªã)
- ããããéèŠãCitusã¯OSSã§ããããªã³ãã¬ã§å©çšã§ãã
- éçºå ã®Citus Dataã¯2019幎ã«ãã€ã¯ããœãããè²·åããŠãããçµå¶ãå®å®ããŠãã
ãªã©å©ç¹ãå€ãããŸãã«æ±ºå®çããšããå°è±¡ã§ãã
ãã€ã¯ããœããã®è€ç°æ°ã«ããã°ãCitus Dataã®åµç«è ããèãã話ã§ãããè²·åææ¡ãåãå ¥ããçç±ã¯ãMicrosoftãOSSã³ãã¥ããã£ãŒã«æãè²¢ç®ããŠãããããªãã¯ã¯ã©ãŠããããã€ããŒã ã£ããããšã®ããšã§ãããšã®ããšããã³ããŒããããŸã§æèšããŠãã以äžã¯ãCitusãOSSãµããŒããäžæ¢ããããšã¯åœåãªãã¯ããã§ãã
Citusã䜿ãã®ã¯Azureãäžçªæ¥œã ãšæããŸãããCitusããªã³ãã¬ã§æäŸãããŠããéãããã®æ°ã«ãªãã°AWSãGCPã®VMäžã§ãCitusã«ããåæ£RDBãæ§ç¯éçšã§ããã¯ããã§ããããã¯ã¯ã©ãŠããã³ããŒã«ããã¯ã€ã³ãããæå¥ãçªç¶ã®å€§å¹ å€äžãã§æ³£ã矜ç®ã«ãªã£ãŠãéããäœå°ãããããšãæå³ããŸãããããã©ãã ãéèŠãªããšãã¯ãITæ¥çã«é·ãæ¹ãªããããåç¥ã§ãããã
ãã¡ã§ã¯ãå šéšãããããã§ãðŠ
æã家ã®èªå® ã¯ã©ã¹ã¿ã¯ã2022幎4ææ«ã®Ubuntu 22.04 LTSã®ãªãªãŒã¹ãåŸ ã£ãŠå šãŠã®ãµãŒããŒOSãå ¥ãæ¿ããäºå®ã§ããããã«åãããŠåæ£ããŒã¿ããŒã¹ãCitusã«å ¥ãæ¿ããããšèããŠããããå šéšãããããã§ããè¿ãå°æ¥PostgreSQLèªäœãåæ£RDBæ©èœãæã€ããã«ãªãããç¥ããŸãããããããåŸ ã£ãŠããããã»ã©ç§ã®å¯¿åœã¯é·ããªãããã§ãã
ããŒã¿æŽ»çšã®èŠ³ç¹ããããåŸæ¥kibanaãPythonãªã©ã«éãããŠããåæããŒã«ãJDBC/ODBC察å¿ã®åçš®BIããŒã«ã«æ¡ããããšãæåŸ ãããŸããç¡åã§å€æ©èœãªPower BI Desktopãªã©ã䜿ããã¯ãã§ãããPostgreSQLèªäœãæã£ãŠãããããªã¢ã©ã€ãºããã¥ãŒãªã©ã®æ©èœãçšããŠRDBåŽã§ã¯ãšãªãé«éåããããšã(ã¹ãã«æ¬¡ç¬¬ã§ã¯)åºæ¥ãã§ãããã
ãããããã€ããŒã¹ã§èšäºãæçš¿ããŠããããšèããŠããŸãã®ã§ãåŒãç¶ããããããé¡ãããŸãã