Linux 学习笔记
2020-05-28
1 安装 VirtualBox
In computing, a virtual machine (VM) is an emulation of a given computer system. Virtual machines operate based on the computer architecture and functions of a real or hypothetical computer, and their implementations may involve specialized hardware, software, or a combination. Wikipedia
Virtualization is an option that allows you to keep your existing operating system and run Ubuntu in a Virtual Machine (VM). This is the method that I recommend for beginning Linux users unless you have an older laptop or PC that you do not use as your primary desktop. I personally run Ubuntu in a VM at home and at school when I teach. I also have a dedicated Ubuntu laptop.
安装 Ubuntu 最好安装一个 VirtualBox 执行虚拟机的思路。
If you want to use Virtualization I recommend VirtualBox. You can download VirtualBox for from from their website https://www.virtualbox.org.
Figure 1.1: 安装VirtualBox
这是甲骨文的软件。下载后，在 Mac 就如图安装其他软件一样进行安装就好了。
2 安装 Ubuntu
This book is a beginner’s guide to installing and using Ubuntu 15.10, the newest version of the popular Linux software. (Vogel 2015)
- Ubuntu 是 linux 系统，
- 其实学习成本最高的在安装上。
Linux 的学习其实在 Mac 上可以完成，因为都是类 Unix 语言。 但是最出名的还是 Ubuntu 系统，下面讲述如何安装 Ubuntu /ʊ’bʊntʊ/ 。
This book is designed for a new Linux or Ubuntu user. Intermediate or advanced Linux users should not purchase this book.
Vogel (2015) 的书是最适合入门的，包含截图一步步完成。 我个人学习的经验是 Linux 貌似最麻烦的是在安装上（个人感觉），因此清晰的安装思路就很值得注意了。
This book is not an exhaustive guide to the Linux operating system. I do not want to overload a new user to Linux with a bunch of advanced topics so my goal is to keep this book relatively simple and geared toward the new Linux user and tackle more advanced topics in other books.
当然选书最后还是要根据的自己的个人偏好，选一本自己认为最入门的书。
I really like Ubuntu’s ease of installation and ease of use once installed. … without having to worry about installing special drivers. For a beginning Linux user you cannot go wrong with Ubuntu.
Ubuntu 的好处可以代码驱动安装很多东西。 反正就是不需要担心各种安装问题。
在安装好 VirtualBox 后，需要设置环境。
Figure 2.1: 内存需要设置在10GB (>7GB)，否则不能安装如下图。
Figure 2.2: 选择从官网下载的iso文件
点击 install Ubuntu 就开始了。
The default option is to erase the disk and install Ubuntu. For most Ubuntu installations this would be the best choice. (Romero 2010)
之后会提示是否需要把盘清空，这里选择默认即可。
安装时间一般会在15分钟左右，相对其他系统安装来说是很快了。
Figure 2.3: 出现这个页面重启即可
2.1 Win 7
Win7 系统一般要求，需要32GB，这比较大，因此忽略。 Win7系统安装虚拟机很卡、重启都很慢，相比较于 Mac 是劣势。
3 安装常用软件
3.1 Git
参考 blog
sudo apt-get install git-coregit config --global user.name "Your Name"
git config --global user.email "email@example.com"tab 可以联想。
3.1.1 数据安全
突然想到如果在一个虚拟机内操作任何，等最后把虚拟机删除了，任何明文密码的问题都没有了。
3.2 R和RStudio
参考 https://mirrors.tuna.tsinghua.edu.cn/CRAN/bin/linux/ubuntu/README.html
服务器一般用 CentOS 多点，UBuntu 是图形界面，相对地，资源消耗多了一点。 CentOS 发音，参考 https://forums.centos.org/viewtopic.php?t=693 Sen-To-S 参考 https://blog.csdn.net/m53931422/article/details/78456850?locationNum=1&fps=1
CentOS
参考 http://qiubio.com/archives/2627
之后安装包，选择镜像或者chooseCRANmirror()选择默认。
Ubuntu
sudo apt-get update
sudo apt-get install r-basesudo -i R查看R版本，记得使用 3.6 最新的版本。 具体实现方式，见 rtask
其中 tidyverse、devtools 包都很大，安装需要谨慎。
RStudio 直接网上下载软件包直接安装。 但是有时候会报错，因为缺少必要的 dependency，这因此推荐用命令安装。
jiaxiang@jiaxiang-VirtualBox:~$ sudo apt-get install gdebi-core
[sudo] password for jiaxiang:
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following NEW packages will be installed:
gdebi-core
0 upgraded, 1 newly installed, 0 to remove and 180 not upgraded.
Need to get 116 kB of archives.
After this operation, 876 kB of additional disk space will be used.
Get:1 http://cn.archive.ubuntu.com/ubuntu bionic/universe amd64 gdebi-core all 0.9.5.7+nmu2 [116 kB]
Fetched 116 kB in 1s (107 kB/s)
Selecting previously unselected package gdebi-core.
(Reading database ... 156839 files and directories currently installed.)
Preparing to unpack .../gdebi-core_0.9.5.7+nmu2_all.deb ...
Unpacking gdebi-core (0.9.5.7+nmu2) ...
Setting up gdebi-core (0.9.5.7+nmu2) ...
Processing triggers for man-db (2.8.3-2ubuntu0.1) ...
jiaxiang@jiaxiang-VirtualBox:~$ sudo gdebi ~/Downloads/rstudio-1.2.1335-amd64
rstudio-1.2.1335-amd64(1).deb rstudio-1.2.1335-amd64.deb
rstudio-1.2.1335-amd64(2).deb
jiaxiang@jiaxiang-VirtualBox:~$ sudo gdebi ~/Downloads/rstudio-1.2.1335-amd64.deb
Reading package lists... Done
Building dependency tree
Reading state information... Done
Reading state information... Done
Requires the installation of the following packages: lib32gcc1 lib32stdc++6 libc6-i386 libclang-6.0-dev libclang-common-6.0-dev libclang-dev libclang1-6.0 libobjc-7-dev libobjc4
RStudio
RStudio is a set of integrated tools designed to help you be more productive with R. It includes a console, syntax-highlighting editor that supports direct code execution, as well as tools for plotting, history, and workspace management.
Do you want to install the software package? [y/N]:y
Fetched 0 B in 0s (0 B/s)
Selecting previously unselected package libc6-i386.
(Reading database ... 156911 files and directories currently installed.)
Preparing to unpack .../0-libc6-i386_2.27-3ubuntu1_amd64.deb ...
Unpacking libc6-i386 (2.27-3ubuntu1) ...
Selecting previously unselected package libobjc4:amd64.
Preparing to unpack .../1-libobjc4_8.3.0-6ubuntu1~18.04_amd64.deb ...
Unpacking libobjc4:amd64 (8.3.0-6ubuntu1~18.04) ...
Selecting previously unselected package libobjc-7-dev:amd64.
Preparing to unpack .../2-libobjc-7-dev_7.4.0-1ubuntu1~18.04_amd64.deb ...
Unpacking libobjc-7-dev:amd64 (7.4.0-1ubuntu1~18.04) ...
Selecting previously unselected package libclang1-6.0:amd64.
Preparing to unpack .../3-libclang1-6.0_1%3a6.0-1ubuntu2_amd64.deb ...
Unpacking libclang1-6.0:amd64 (1:6.0-1ubuntu2) ...
Selecting previously unselected package lib32gcc1.
Preparing to unpack .../4-lib32gcc1_1%3a8.3.0-6ubuntu1~18.04_amd64.deb ...
Unpacking lib32gcc1 (1:8.3.0-6ubuntu1~18.04) ...
Selecting previously unselected package lib32stdc++6.
Preparing to unpack .../5-lib32stdc++6_8.3.0-6ubuntu1~18.04_amd64.deb ...
Unpacking lib32stdc++6 (8.3.0-6ubuntu1~18.04) ...
Selecting previously unselected package libclang-common-6.0-dev.
Preparing to unpack .../6-libclang-common-6.0-dev_1%3a6.0-1ubuntu2_amd64.deb ...
Unpacking libclang-common-6.0-dev (1:6.0-1ubuntu2) ...
Selecting previously unselected package libclang-6.0-dev.
Preparing to unpack .../7-libclang-6.0-dev_1%3a6.0-1ubuntu2_amd64.deb ...
Unpacking libclang-6.0-dev (1:6.0-1ubuntu2) ...
Selecting previously unselected package libclang-dev.
Preparing to unpack .../8-libclang-dev_1%3a6.0-41~exp5~ubuntu1_amd64.deb ...
Unpacking libclang-dev (1:6.0-41~exp5~ubuntu1) ...
Setting up libobjc4:amd64 (8.3.0-6ubuntu1~18.04) ...
Setting up libclang1-6.0:amd64 (1:6.0-1ubuntu2) ...
Setting up libc6-i386 (2.27-3ubuntu1) ...
Setting up libobjc-7-dev:amd64 (7.4.0-1ubuntu1~18.04) ...
Processing triggers for libc-bin (2.27-3ubuntu1) ...
Setting up lib32gcc1 (1:8.3.0-6ubuntu1~18.04) ...
Setting up lib32stdc++6 (8.3.0-6ubuntu1~18.04) ...
Setting up libclang-common-6.0-dev (1:6.0-1ubuntu2) ...
Setting up libclang-6.0-dev (1:6.0-1ubuntu2) ...
Setting up libclang-dev (1:6.0-41~exp5~ubuntu1) ...
Processing triggers for libc-bin (2.27-3ubuntu1) ...
Selecting previously unselected package rstudio.
(Reading database ... 158230 files and directories currently installed.)
Preparing to unpack .../rstudio-1.2.1335-amd64.deb ...
Unpacking rstudio (1.2.1335) ...
Setting up rstudio (1.2.1335) ...
Processing triggers for shared-mime-info (1.9-2) ...
Processing triggers for gnome-menus (3.13.3-11ubuntu1.1) ...
Processing triggers for desktop-file-utils (0.23-1ubuntu3.18.04.2) ...
Processing triggers for mime-support (3.60ubuntu1) ...
Processing triggers for hicolor-icon-theme (0.17-2) ...系统会提示安装 dependency。
并且这里推荐
Your best bet is to use
gdebito do RStudio installations. It automatically resolves dependencies (unlikedpkg).sudo apt-get install gdebi-core sudo gdebi rstudio-0.99.902-i386.debIn fact, that’s what RStudio themselves recommends for rstudio-server installation. askubuntu
3.3 conda
参考 https://conda.io/projects/conda/en/latest/user-guide/install/linux.html
先下载 https://docs.conda.io/en/latest/miniconda.html 再安装 https://conda.io/projects/conda/en/latest/user-guide/install/linux.html
bash Miniconda3-latest-Linux-x86_64.sh重启 Terminal，然后就可以用了。
3.4 tidyverse
一般安装都会报错，需要安装一些 dependency。
sudo apt-get install libssl-dev libxml2-dev libcurl4-openssl-devSo I followed each prompt, installing the required deb, which came up to Stack Overflow
5 定时任务
crontab 是 linux 一个定时任务的包。
参考 Hoffman (2017) 和 Stack Overflow 和 cronR 包和 www.thesitewizard.com 和 WeChat Article 直接在 crontab 中设定
相关竞品，如 taskscheduleR 是用来在 Windows 上实现定时任务的，但是失灵率很高。
5.1 基本操作
查看，方便复制
新增，我觉得差不多了，我需要知道一次性的 job 怎么更好的建立。
在 MacOS 测试。
## cronR job
## id: delete_job
## tags: housekeeping
## desc: Delete unneccessary files hourly. 4
2 * * * * /Library/Frameworks/R.framework/Resources/bin/Rscript '/Users/vija/Downloads/work/imp_rmd/R/delete-tools.R' >> '/Users/vija/Downloads/work/imp_rmd/R/delete-tools.log' 2>&15.2 重要备注
目前测试 linux 上可以跑通。
cronR包- Terminal 内
Rscript R/main.R执行成功 here::here()函数使用前需要用setwd设置好项目的绝对路径，如setwd("/home/jiaxiang/Documents/job-automation")- 可以自动 push，设置好 SSH Keys 即可。
- 目前只打开了 Ubuntu，锁住电脑的前提下也可以。
5.3 完全磁盘访问权限
根据 www.jianshu.com 设置下 RStudio Terminals 的权限即可。
命令行终端没有权限，需要打开 Mac 的系统偏好设置 > 安全性与隐私 > 隐私 > 完全磁盘访问权限。
5.5 使用 Rmd 进行管理
crontab_list.Rmd 作为配置文件进行管理，通过 crontab output/crontab_list.Rmd 进行覆盖更新。
5.6 pandoc 路径设置
5.7 手动增加日志
测试命令行执行成功
这样执行就是手动增加日志。
5.8 blogdown 自动刷新
This function recompiles all R Markdown files by default, even if the output files are newer than the source files. If you want to build the site without rebuilding all R Markdown files, you should use
hugo_build()instead.
这样就可以不被 preview 干扰，不能跑 job 了。
## crontab job
## id: serve_blog
## tags: automation
## desc: serve site daily.
0 22 * * * /Library/Frameworks/R.framework/Resources/bin/Rscript '/Users/vija/Downloads/work/blog_181126/R/serve_site_automation.R' >> '/Users/vija/Downloads/work/blog_181126/R/serve_site_automation.log' 2>&1- 博客的更新可以也使用 crontab，不需要我自己 serve site 很麻烦。
- 如何在 cmd 批量 git push 和 git pull
5.9 RStudio API
执行过程中避免 rstudioapi::的函数，因为会报错。
错误: RStudio not running
停止执行5.10 固定一段时间更新
参考 https://www.computerhope.com/unix/ucrontab.htm
00 09-18 * * * /home/carl/hourly-archive.sh
0 9,18 * * Mon /home/wendy/script.sh
0 0-4,8-12 * * Mon /home/wendy/script.shLists are allowed. A list is a set of numbers (or ranges) separated by commas. Examples: “1,2,5,9”, “0-4,8-12”.
6 快捷方式
Ubuntu 可以全屏，cmd + F 全屏切换。前置需要先点击 增强功能，然后在 Ubuntu 的桌面上打开 VBox_GAs_6.0.8，就好了。
tab + alt 切换，类似于 Win7
Ubuntu 切换输入法 shift
输入法会自动改变默认文件夹的名字，如图。
Figure 6.1: 自动改变默认文件夹的名字
7 主要命令
7.1 查看系统版本
$ cat /proc/version
Linux version 4.18.0-15-generic (buildd@lcy01-amd64-029) (gcc version 7.3.0 (Ubuntu 7.3.0-16ubuntu3)) #16~18.04.1-Ubuntu SMP Thu Feb 7 14:06:04 UTC 20197.2 关机
7.3 解压
7.4 du 命令
7.5 du help 文档
```bash
du --help
Usage: /Rtools/bin/du [OPTION]... [FILE]...
or: /Rtools/bin/du [OPTION]... --files0-from=F
Summarize disk usage of the set of FILEs, recursively for directories.
Mandatory arguments to long options are mandatory for short options too.
-0, --null end each output line with NUL, not newline
-a, --all write counts for all files, not just directories
--apparent-size print apparent sizes, rather than disk usage; although
the apparent size is usually smaller, it may be
larger due to holes in ('sparse') files, internal
fragmentation, indirect blocks, and the like
-B, --block-size=SIZE scale sizes by SIZE before printing them; e.g.,
'-BM' prints sizes in units of 1,048,576 bytes;
see SIZE format below
-b, --bytes equivalent to '--apparent-size --block-size=1'
-c, --total produce a grand total
-D, --dereference-args dereference only symlinks that are listed on the
command line
-d, --max-depth=N print the total for a directory (or file, with --all)
only if it is N or fewer levels below the command
line argument; --max-depth=0 is the same as
--summarize
--files0-from=F summarize disk usage of the
NUL-terminated file names specified in file F;
if F is -, then read names from standard input
-H equivalent to --dereference-args (-D)
-h, --human-readable print sizes in human readable format (e.g., 1K 234M 2G)
--inodes list inode usage information instead of block usage
-k like --block-size=1K
-L, --dereference dereference all symbolic links
-l, --count-links count sizes many times if hard linked
-m like --block-size=1M
-P, --no-dereference don't follow any symbolic links (this is the default)
-S, --separate-dirs for directories do not include size of subdirectories
--si like -h, but use powers of 1000 not 1024
-s, --summarize display only a total for each argument
-t, --threshold=SIZE exclude entries smaller than SIZE if positive,
or entries greater than SIZE if negative
--time show time of the last modification of any file in the
directory, or any of its subdirectories
--time=WORD show time as WORD instead of modification time:
atime, access, use, ctime or status
--time-style=STYLE show times using STYLE, which can be:
full-iso, long-iso, iso, or +FORMAT;
FORMAT is interpreted like in 'date'
-X, --exclude-from=FILE exclude files that match any pattern in FILE
--exclude=PATTERN exclude files that match PATTERN
-x, --one-file-system skip directories on different file systems
--help display this help and exit
--version output version information and exit
Display values are in units of the first available SIZE from --block-size,
and the DU_BLOCK_SIZE, BLOCK_SIZE and BLOCKSIZE environment variables.
Otherwise, units default to 1024 bytes (or 512 if POSIXLY_CORRECT is set).
The SIZE argument is an integer and optional unit (example: 10K is 10*1024).
Units are K,M,G,T,P,E,Z,Y (powers of 1024) or KB,MB,... (powers of 1000).
GNU coreutils online help: <http://www.gnu.org/software/coreutils/>
Report du translation bugs to <http://translationproject.org/team/>
Full documentation at: <http://www.gnu.org/software/coreutils/du>
or available locally via: info '(coreutils) du invocation'
```7.6 查询路径下所有子路径大小
以项目 learn_rmd 为例
$ du -h --max-depth=1
6.7M ./.git
2.7M ./.Rproj.user
966K ./analysis
722K ./analysis-2
2.0K ./code
6.0K ./cookbook_cache
1.2M ./cookbook_files
1.0K ./data
5.1M ./docs
72K ./figure
3.0M ./libs
2.0K ./output
2.0K ./R
0 ./refs
6.0K ./template
0 ./tmp
1.4M ./xaringan
22M .可以知道总文件夹大小22M .，
./.git可以知道 git 文件夹大小。
8 调用 Python
这个 Terminal 和 MacOS 里面的一致的，只是系统不一样。
$ python --version #查看版本
Python 3.7.3
$ python
Python 3.7.3 (default, Apr 24 2019, 15:29:51) [MSC v.1915 64 bit (AMD64)] :: Anaconda, Inc. on win32
Warning:
This Python interpreter is in a conda environment, but the environment has
not been activated. Libraries may fail to load. To activate this environment
please see https://conda.io/activation
Type "help", "copyright", "credits" or "license" for more information.
>>>以上就是调用成功，$代表命令行输入，其他为输出。
下面就可以正常调用和输入了
就和正常的 console 很像，比如你在 PyCharm 或者 RStudio 的 console 调用一样，或者在 Python 原生的 IDE 里面一回事。
9 后台执行
命令后面加个&试试，就是命令后台执行。
10 其他
Figure 10.1: 路径和 Mac 很像
版本控制，就是说里面可以独立做一些事情，哪怕明文密码都没关系，反正用完删掉就好。 另外我需要用一个定时服务 cron，这个linux 做的最好，win7不稳定。
在 Other Location 里面查看 电脑内存。
参考文献
Hoffman, Chris. 2017. “How to Schedule Tasks on Linux: An Introduction to Crontab Files.” How-To Geek. 2017. https://www.howtogeek.com/101288/how-to-schedule-tasks-on-linux-an-introduction-to-crontab-files/.
Romero, Alfonso. 2010. VirtualBox 3.1: Beginner’s Guide. Packt Publishing.
Vogel, Matthew. 2015. Get Started with Ubuntu 15.10. Matthew Vogel.