文章目录
- weed命令
- 翻译
- weed server命令
- 翻译
- weed filer命令
- docker容器运行示例(docker run命令)
- 以下是一个在单个容器启动master、volume、filer服务的示例
- weed server 容器debug
版本:3.47
从官方镜像docker容器里扒下来的,只扒了自己用到的一些参数,想扒其他的参数,请在评论区留言。
weed命令
SeaweedFS: store billions of files and serve them fast!Usage:weed command [arguments]The commands are:autocomplete install autocompleteautocomplete.uninstall uninstall autocompletebackup incrementally backup a volume to local folderbenchmark benchmark by writing millions of files and reading them outcompact run weed tool compact on volume filedownload download files by file idexport list or export files from one volume data filefiler start a file server that points to a master server, or a list of master serversfiler.backup resume-able continuously replicate files from a SeaweedFS cluster to another location defined in replication.tomlfiler.cat copy one file to localfiler.copy copy one or a list of files to a filer folderfiler.meta.backup continuously backup filer meta data changes to anther filer store specified in a backup_filer.tomlfiler.meta.tail see continuous changes on a filerfiler.remote.gateway resumable continuously write back bucket creation, deletion, and other local updates to remote object storefiler.remote.sync resumable continuously write back updates to remote storagefiler.replicate replicate file changes to another destinationfiler.sync resumable continuous synchronization between two active-active or active-passive SeaweedFS clustersfix run weed tool fix on files or whole folders to recreate index file(s) if corruptedfuse Allow use weed with linux's mount commandiam start a iam API compatible servermaster start a master servermaster.follower start a master followermount mount weed filer to a directory as file system in userspace(FUSE)mq.broker <WIP> start a message queue brokers3 start a s3 API compatible server that is backed by a filerscaffold generate basic configuration filesserver start a master server, a volume server, and optionally a filer and a S3 gatewayshell run interactive administrative commandsupdate get latest or specific version from https://github.com/seaweedfs/seaweedfsupload upload one or a list of filesversion print SeaweedFS versionvolume start a volume serverwebdav start a webdav server that is backed by a filerUse "weed help [command]" for more information about a command.For Logging, use "weed [logging_options] [command]". The logging options are:-alsologtostderrlog to standard error as well as files (default true)-config_dir valuedirectory with toml configuration files-log_backtrace_at valuewhen logging hits line file:N, emit a stack trace-logdir stringIf non-empty, write log files in this directory-logtostderrlog to standard error instead of files-options stringa file of command line options, each line in optionName=optionValue format-stderrthreshold valuelogs at or above this threshold go to stderr-v valuelog levels [0|1|2|3|4], default to 0-vmodule valuecomma-separated list of pattern=N settings for file-filtered logging
翻译
这是一个关于SeaweedFS的命令行工具的文档,SeaweedFS是一个分布式的文件系统。该文档列出了SeaweedFS命令行工具中可用的命令,包括:- autocomplete:安装或卸载自动补全插件
- backup:将卷增量备份到本地文件夹
- benchmark:通过写入数百万个文件并读取它们来进行基准测试
- compact:在卷文件上运行weed工具压缩
- download:通过文件ID下载文件
- export:从一个卷数据文件中列出或导出文件
- filer:启动一个文件服务器,指向主服务器或主服务器列表
- filer.backup:从SeaweedFS集群连续复制文件到在replication.toml中定义的另一个位置
- filer.cat:将一个文件复制到本地
- filer.copy:将一个或多个文件复制到filer文件夹
- filer.meta.backup:连续备份filer元数据更改到在backup_filer.toml中指定的另一个filer存储
- filer.meta.tail:查看filer上的连续更改
- filer.remote.gateway:可恢复地连续将存储桶创建、删除和其他本地更新写回远程对象存储
- filer.remote.sync:可恢复地连续将更新写回远程存储
- filer.replicate:将文件更改复制到另一个目标
- filer.sync:在两个活动-活动或活动-被动SeaweedFS集群之间进行可恢复的连续同步
- fix:在文件或整个文件夹上运行weed工具修复,以重新创建索引文件(如果已损坏)
- fuse:允许使用linux的mount命令与weed一起使用
- iam:启动一个兼容IAM API的服务器
- master:启动一个主服务器
- master.follower:启动一个主服务器的follower
- mount:将weed filer挂载到目录中作为用户空间文件系统(FUSE)
- mq.broker:启动一个消息队列代理(WIP)
- s3:启动一个由filer支持的S3 API兼容服务器
- scaffold:生成基本配置文件
- server:启动一个主服务器、一个卷服务器,以及可选的filer和S3网关
- shell:运行交互式管理命令
- update:从https://github.com/seaweedfs/seaweedfs获取最新或特定版本
- upload:上传一个或多个文件
- version:打印SeaweedFS版本
- volume:启动一个卷服务器
- webdav:启动一个由filer支持的WebDAV服务器文档还提供了有关日志记录的选项,可以使用"weed [logging_options] [command]"来记录日志。日志选项包括:- -alsologtostderr:将日志记录到标准错误和文件中(默认为true)
- -config_dir:包含toml配置文件的目录
- -log_backtrace_at:当日志命中行文件:N时,输出堆栈跟踪
- -logdir:如果不为空,则在此目录中写入日志文件
- -logtostderr:将日志记录到标准错误而不是文件中
- -options:一个命令行选项文件,每行以optionName=optionValue格式
- -stderrthreshold:将日志记录到标准错误的阈值
- -v:日志级别[0|1|2|3|4],默认为0
- -vmodule:文件过滤日志的模式=N设置的逗号分隔列表
weed server命令
/data # weed server --help
Example: weed server -dir=/tmp -volume.max=5 -ip=server_name
Default Usage:-cpuprofile stringcpu profile output file-dataCenter stringcurrent volume server's data center name-debugserves runtime profiling data, e.g., http://localhost:6060/debug/pprof/goroutine?debug=2-debug.port inthttp port for debugging (default 6060)-dir stringdirectories to store data files. dir[,dir]... (default "/tmp")-disableHttpdisable http requests, only gRPC operations are allowed.-filerwhether to start filer-filer.collection stringall data will be stored in this collection-filer.concurrentUploadLimitMB intlimit total concurrent upload size (default 64)-filer.defaultReplicaPlacement stringdefault replication type. If not specified, use master setting.-filer.dirListLimit intlimit sub dir listing size (default 1000)-filer.disableDirListingturn off directory listing-filer.disk string[hdd|ssd|<tag>] hard drive or solid state drive or any tag-filer.downloadMaxMBps intdownload max speed for each download request, in MB per second-filer.encryptVolumeDataencrypt data on volume servers-filer.filerGroup stringshare metadata with other filers in the same filerGroup-filer.localSocket stringdefault to /tmp/seaweedfs-filer-<port>.sock-filer.maxMB intsplit files larger than the limit (default 4)-filer.port intfiler server http listen port (default 8888)-filer.port.grpc intfiler server grpc listen port-filer.port.public intfiler server public http listen port-filer.saveToFilerLimit intSmall files smaller than this limit can be cached in filer store.-filer.ui.deleteDirenable filer UI show delete directory button (default true)-iamwhether to start IAM service-iam.port intiam server http listen port (default 8111)-idleTimeout intconnection idle seconds (default 30)-ip stringip or server name, also used as identifier (default "172.17.0.5")-ip.bind stringip address to bind to. If empty, default to same as -ip option.-masterwhether to start master server (default true)-master.defaultReplication stringDefault replication type if not specified.-master.dir stringdata directory to store meta data, default to same as -dir specified-master.electionTimeout durationelection timeout of master servers (default 10s)-master.garbageThreshold floatthreshold to vacuum and reclaim spaces (default 0.3)-master.heartbeatInterval durationheartbeat interval of master servers, and will be randomly multiplied by [1, 1.25) (default 300ms)-master.metrics.address stringPrometheus gateway address-master.metrics.intervalSeconds intPrometheus push interval in seconds (default 15)-master.peers stringall master nodes in comma separated ip:masterPort list-master.port intmaster server http listen port (default 9333)-master.port.grpc intmaster server grpc listen port-master.raftHashicorpuse hashicorp raft-master.resumeStateresume previous state on start master server-master.volumePreallocatePreallocate disk space for volumes.-master.volumeSizeLimitMB uintMaster stops directing writes to oversized volumes. (default 30000)-memprofile stringmemory profile output file-metricsPort intPrometheus metrics listen port-mq.brokerwhether to start message queue broker-mq.broker.port intmessage queue broker gRPC listen port (default 17777)-options stringa file of command line options, each line in optionName=optionValue format-rack stringcurrent volume server's rack name-s3whether to start S3 gateway-s3.allowDeleteBucketNotEmptyallow recursive deleting all entries along with bucket (default true)-s3.allowEmptyFolderallow empty folders (default true)-s3.auditLogConfig stringpath to the audit log config file-s3.cert.file stringpath to the TLS certificate file-s3.config stringpath to the config file-s3.domainName stringsuffix of the host name in comma separated list, {bucket}.{domainName}-s3.key.file stringpath to the TLS private key file-s3.port ints3 server http listen port (default 8333)-s3.port.grpc ints3 server grpc listen port-volumewhether to start volume server (default true)-volume.compactionMBps intlimit compaction speed in mega bytes per second-volume.concurrentDownloadLimitMB intlimit total concurrent download size (default 64)-volume.concurrentUploadLimitMB intlimit total concurrent upload size (default 64)-volume.dir.idx stringdirectory to store .idx files-volume.disk string[hdd|ssd|<tag>] hard drive or solid state drive or any tag-volume.fileSizeLimitMB intlimit file size to avoid out of memory (default 256)-volume.hasSlowRead<experimental> if true, this prevents slow reads from blocking other requests, but large file read P99 latency will increase. (default true)-volume.images.fix.orientationAdjust jpg orientation when uploading.-volume.index stringChoose [memory|leveldb|leveldbMedium|leveldbLarge] mode for memory~performance balance. (default "memory")-volume.index.leveldbTimeout intalive time for leveldb (default to 0). If leveldb of volume is not accessed in ldbTimeout hours, it will be off loaded to reduce opened files and memory consumption.-volume.inflightUploadDataTimeout durationinflight upload data wait timeout of volume servers (default 1m0s)-volume.max stringmaximum numbers of volumes, count[,count]... If set to zero, the limit will be auto configured as free disk space divided by volume size. (default "8")-volume.minFreeSpace stringmin free disk space (value<=100 as percentage like 1, other as human readable bytes, like 10GiB). Low disk space will mark all volumes as ReadOnly.-volume.minFreeSpacePercent stringminimum free disk space (default to 1%). Low disk space will mark all volumes as ReadOnly (deprecated, use minFreeSpace instead). (default "1")-volume.port intvolume server http listen port (default 8080)-volume.port.grpc intvolume server grpc listen port-volume.port.public intvolume server public port-volume.pprofenable pprof http handlers. precludes --memprofile and --cpuprofile-volume.preStopSeconds intnumber of seconds between stop send heartbeats and stop volume server (default 10)-volume.publicUrl stringpublicly accessible address-volume.readBufferSizeMB int<experimental> larger values can optimize query performance but will increase some memory usage,Use with hasSlowRead normally (default 4)-volume.readMode string[local|proxy|redirect] how to deal with non-local volume: 'not found|read in remote node|redirect volume location'. (default "proxy")-webdavwhether to start WebDAV gateway-webdav.cacheCapacityMB intlocal cache capacity in MB-webdav.cacheDir stringlocal cache directory for file chunks (default "/tmp")-webdav.cert.file stringpath to the TLS certificate file-webdav.collection stringcollection to create the files-webdav.disk string[hdd|ssd|<tag>] hard drive or solid state drive or any tag-webdav.filer.path stringuse this remote path from filer server (default "/")-webdav.key.file stringpath to the TLS private key file-webdav.port intwebdav server http listen port (default 7333)-webdav.replication stringreplication to create the files-whiteList stringcomma separated Ip addresses having write permission. No limit if empty.
Description:start both a volume server to provide storage spacesand a master server to provide volume=>location mapping service and sequence number of file idsThis is provided as a convenient way to start both volume server and master server.The servers acts exactly the same as starting them separately.So other volume servers can connect to this master server also.Optionally, a filer server can be started.Also optionally, a S3 gateway can be started.
翻译
weed server的参数翻译如下:默认用法:- -cpuprofile string:CPU分析输出文件
- -dataCenter string:当前卷服务器的数据中心名称
- -debug:提供运行时分析数据,例如http://localhost:6060/debug/pprof/goroutine?debug=2
- -debug.port int:调试的http端口(默认6060)
- -dir string:存储数据文件的目录。dir[,dir]...(默认“/tmp”)
- -disableHttp:禁用http请求,只允许gRPC操作。
- -filer:是否启动filer
- -filer.collection string:所有数据将存储在此集合中
- -filer.concurrentUploadLimitMB int:限制总并发上传大小(默认64)
- -filer.defaultReplicaPlacement string:默认复制类型。如果未指定,则使用主设置。
- -filer.dirListLimit int:限制子目录列表大小(默认1000)
- -filer.disableDirListing:关闭目录列表
- -filer.disk string:[hdd|ssd|<tag>]硬盘或固态硬盘或任何标签
- -filer.downloadMaxMBps int:每个下载请求的下载最大速度,以MB每秒为单位
- -filer.encryptVolumeData:在卷服务器上加密数据
- -filer.filerGroup string:与同一filerGroup中的其他filer共享元数据
- -filer.localSocket string:默认为/tmp/seaweedfs-filer-<port>.sock
- -filer.maxMB int:拆分大于限制的文件(默认4)
- -filer.port int:filer服务器http监听端口(默认8888)
- -filer.port.grpc int:filer服务器grpc监听端口
- -filer.port.public int:filer服务器公共http监听端口
- -filer.saveToFilerLimit int:小于此限制的小文件可以缓存在filer存储中。
- -filer.ui.deleteDir:启用filer UI显示删除目录按钮(默认为true)
- -iam:是否启动IAM服务
- -iam.port int:iam服务器http监听端口(默认8111)
- -idleTimeout int:连接空闲秒数(默认30)
- -ip string:IP或服务器名称,也用作标识符(默认“172.17.0.5”)
- -ip.bind string:要绑定的IP地址。如果为空,则默认为与-ip选项相同。
- -master:是否启动主服务器(默认为true)
- -master.defaultReplication string:如果未指定,则默认复制类型。
- -master.dir string:存储元数据的数据目录,默认与指定的-dir相同
- -master.electionTimeout duration:主服务器的选举超时(默认10s)
- -master.garbageThreshold float:清理和回收空间的阈值(默认0.3)
- -master.heartbeatInterval duration:主服务器的心跳间隔,并将随机乘以[1,1.25)(默认300ms)
- -master.metrics.address string:Prometheus网关地址
- -master.metrics.intervalSeconds int:Prometheus推送间隔(默认15秒)
- -master.peers string:逗号分隔的ip:masterPort列表中的所有主节点
- -master.port int:主服务器http监听端口(默认9333)
- -master.port.grpc int:主服务器grpc监听端口
- -master.raftHashicorp:使用hashicorp raft
- -master.resumeState:在启动主服务器时恢复以前的状态
- -master.volumePreallocate:为卷预分配磁盘空间。
- -master.volumeSizeLimitMB uint:主服务器停止将写入定向到超大卷。(默认30000)
- -memprofile string:内存分析输出文件
- -metricsPort int:Prometheus指标监听端口
- -mq.broker:是否启动消息队列代理
- -mq.broker.port int:消息队列代理gRPC监听端口(默认17777)
- -options string:命令行选项文件,每行以optionName=optionValue格式
- -rack string:当前卷服务器的机架名称
- -s3是否启动S3网关-s3.allowDeleteBucketNotEmpty允许递归删除所有条目以及存储桶(默认为true)-s3.allowEmptyFolder允许空文件夹(默认为true)-s3.auditLogConfig string审计日志配置文件的路径-s3.cert.file stringTLS证书文件的路径-s3.config string配置文件的路径-s3.domainName string主机名后缀,以逗号分隔,{bucket}.{domainName}-s3.key.file stringTLS私钥文件的路径-s3.port ints3服务器的http监听端口(默认为8333)-s3.port.grpc ints3服务器的grpc监听端口-volume是否启动卷服务器(默认为true)-volume.compactionMBps int限制压缩速度,以兆字节每秒为单位-volume.concurrentDownloadLimitMB int限制总并发下载大小(默认为64)-volume.concurrentUploadLimitMB int限制总并发上传大小(默认为64)-volume.dir.idx string存储.idx文件的目录-volume.disk string[hdd|ssd|<tag>] 硬盘或固态硬盘或任何标签-volume.fileSizeLimitMB int限制文件大小以避免内存不足(默认为256)-volume.hasSlowRead<实验性>如果为true,则可以防止缓慢的读取阻塞其他请求,但大文件读取P99延迟将增加。(默认为true)-volume.images.fix.orientation上传时调整jpg方向。-volume.index string选择[内存|leveldb|leveldbMedium|leveldbLarge]模式以实现内存~性能平衡。(默认为"memory")-volume.index.leveldbTimeout intleveldb的存活时间(默认为0)。如果卷的leveldb在ldbTimeout小时内未被访问,则会卸载它以减少打开的文件和内存消耗。-volume.inflightUploadDataTimeout duration卷服务器的inflight上传数据等待超时时间(默认为1m0s)-volume.max string最大卷数,count[,count]...如果设置为零,则限制将自动配置为可用磁盘空间除以卷大小。(默认为"8")-volume.minFreeSpace string最小可用磁盘空间(value<=100表示百分比,如1,其他表示人类可读的字节,如10GiB)。低磁盘空间将将所有卷标记为只读。-volume.minFreeSpacePercent string最小可用磁盘空间(默认为1%)。低磁盘空间将将所有卷标记为只读(已弃用,请改用minFreeSpace)。 (默认为"1")-volume.port int卷服务器的http监听端口(默认为8080)-volume.port.grpc int卷服务器的grpc监听端口-volume.port.public int卷服务器的公共端口-volume.pprof启用pprof http处理程序。排除--memprofile和--cpuprofile-volume.preStopSeconds int停止发送心跳和停止卷服务器之间的秒数(默认为10)-volume.publicUrl string公开访问地址-volume.readBufferSizeMB int<实验性> 较大的值可以优化查询性能,但会增加一些内存使用量,通常与hasSlowRead一起使用(默认值为4)-volume.readMode string[local|proxy|redirect] 处理非本地卷的方式:'未找到|在远程节点中读取|重定向卷位置'。(默认值为“proxy”)-webdav是否启动WebDAV网关-webdav.cacheCapacityMB int本地缓存容量(以MB为单位)-webdav.cacheDir string文件块的本地缓存目录(默认为“/tmp”)-webdav.cert.file stringTLS证书文件的路径-webdav.collection string要创建文件的集合-webdav.disk string[hdd|ssd|<tag>] 硬盘或固态硬盘或任何标签-webdav.filer.path string使用filer服务器上的此远程路径(默认为“/”)-webdav.key.file stringTLS私钥文件的路径-webdav.port intwebdav服务器http监听端口(默认为7333)-webdav.replication string要创建文件的复制品-whiteList string具有写入权限的逗号分隔IP地址。如果为空,则没有限制。描述:启动一个卷服务器以提供存储空间,并启动一个主服务器以提供卷=>位置映射服务和文件ID的序列号。这是一种方便的方式,可以同时启动卷服务器和主服务器。服务器的行为与分别启动它们完全相同。因此,其他卷服务器也可以连接到此主服务器。可选地,可以启动filer服务器。还可以选择启动S3网关。
weed filer命令
Example: weed filer -port=8888 -master=<ip:port>[,<ip:port>]*
Default Usage:-collection stringall data will be stored in this default collection-concurrentUploadLimitMB intlimit total concurrent upload size (default 128)-dataCenter stringprefer to read and write to volumes in this data center-debugserves runtime profiling data, e.g., http://localhost:<debug.port>/debug/pprof/goroutine?debug=2-debug.port inthttp port for debugging (default 6060)-defaultReplicaPlacement stringdefault replication type. If not specified, use master setting.-defaultStoreDir stringif filer.toml is empty, use an embedded filer store in the directory (default ".")-dirListLimit intlimit sub dir listing size (default 100000)-disableDirListingturn off directory listing-disableHttpdisable http request, only gRpc operations are allowed-disk string[hdd|ssd|<tag>] hard drive or solid state drive or any tag-downloadMaxMBps intdownload max speed for each download request, in MB per second-encryptVolumeDataencrypt data on volume servers-filerGroup stringshare metadata with other filers in the same filerGroup-iamwhether to start IAM service-iam.ip stringiam server http listen ip address (default "172.17.0.5")-iam.port intiam server http listen port (default 8111)-ip stringfiler server http listen ip address (default "172.17.0.5")-ip.bind stringip address to bind to. If empty, default to same as -ip option.-localSocket stringdefault to /tmp/seaweedfs-filer-<port>.sock-master stringcomma-separated master servers (default "localhost:9333")-maxMB intsplit files larger than the limit (default 4)-metricsPort intPrometheus metrics listen port-options stringa file of command line options, each line in optionName=optionValue format-port intfiler server http listen port (default 8888)-port.grpc intfiler server grpc listen port-port.readonly intreadonly port opened to public-rack stringprefer to write to volumes in this rack-s3whether to start S3 gateway-s3.allowDeleteBucketNotEmptyallow recursive deleting all entries along with bucket (default true)-s3.allowEmptyFolderallow empty folders (default true)-s3.auditLogConfig stringpath to the audit log config file-s3.cert.file stringpath to the TLS certificate file-s3.config stringpath to the config file-s3.dataCenter stringprefer to read and write to volumes in this data center-s3.domainName stringsuffix of the host name in comma separated list, {bucket}.{domainName}-s3.key.file stringpath to the TLS private key file-s3.port ints3 server http listen port (default 8333)-s3.port.grpc ints3 server grpc listen port-saveToFilerLimit intfiles smaller than this limit will be saved in filer store-ui.deleteDirenable filer UI show delete directory button (default true)-webdavwhether to start webdav gateway-webdav.cacheCapacityMB intlocal cache capacity in MB-webdav.cacheDir stringlocal cache directory for file chunks (default "/tmp")-webdav.cert.file stringpath to the TLS certificate file-webdav.collection stringcollection to create the files-webdav.disk string[hdd|ssd|<tag>] hard drive or solid state drive or any tag-webdav.filer.path stringuse this remote path from filer server (default "/")-webdav.key.file stringpath to the TLS private key file-webdav.port intwebdav server http listen port (default 7333)-webdav.replication stringreplication to create the files
Description:start a file server which accepts REST operation for any files.//create or overwrite the file, the directories /path/to will be automatically createdPOST /path/to/file//get the file contentGET /path/to/file//create or overwrite the file, the filename in the multipart request will be usedPOST /path/to///return a json format subdirectory and files listingGET /path/to/The configuration file "filer.toml" is read from ".", "$HOME/.seaweedfs/", "/usr/local/etc/seaweedfs/", or "/etc/seaweedfs/", in that order.If the "filer.toml" is not found, an embedded filer store will be created under "-defaultStoreDir".The example filer.toml configuration file can be generated by "weed scaffold -config=filer"Supported Filer Stores:arangodbcassandraetcdhbaseleveldbleveldb2leveldb3mongodbmysqlmysql2postgrespostgres2redisredis2redis2_sentinelredis3redis3_sentinelredis_clusterredis_cluster2redis_cluster3
docker容器运行示例(docker run命令)
以下是一个在单个容器启动master、volume、filer服务的示例
docker run \-d -i -t \--name $CONTAINER_NAME \-p $MASTER_PORT:9333 \-p $FILER_PORT:8888 \-v $SCRIPT_LOCATION/mount/masterVolumeFiler/data/:/data/ \-v /etc/localtime:/etc/localtime:ro \$IMAGE_NAME:$IMAGE_TAG \server -filer
其中server
就是前面提到的weed命令参数之一,表示在一个单一容器中同时启动master和volume服务,在我们的docker run命令中,它接在docker镜像:tag的后面;而-filer
是一个weed server的参数的子参数,表示同时启动一个filer服务,这些参数都可以通过我们上面的参数表查到。
weed server 容器debug
需要映射出6060端口,同时加上-debug
参数,如:server -filer -debug
然后容器起起来后,浏览器访问:
http://192.168.1.140:6060/debug/pprof/
点击就可访问各模块日志: