ELK Stack分布式日志分析

ELK Stack 包括 Elasticsearch、Logstash、Kibana 和 Beats 。能够安全可靠地获取任何来源、任何格式的数据，然后实时地对数据进行搜索、分析和可视化。

  • Elasticsearch 是一个分布式、RESTful 风格的搜索和数据分析引擎，能够解决不断涌现出的各种用例。 作为 Elastic Stack 的核心，它集中存储您的数据，帮助您发现意料之中以及意料之外的情况。
  • Kibana 可以对自己的 Elasticsearch 进行可视化，还可以在 Elastic Stack 中进行导航，这样您便可以进行各种操作了，从跟踪查询负载，到理解请求如何流经您的整个应用，都能轻松完成。
  • Logstash 是免费且开放的服务器端数据处理管道，能够从多个来源采集数据，转换数据，然后将数据发送到您最喜欢的“存储库”中。
  • Beats 是一个免费且开放的平台，集合了多种单一用途数据采集器。它们从成百上千或成千上万台机器和系统向 Logstash 或 Elasticsearch 发送数据。包括 Filebeat、Metricbeat、Packetbeat、Winlogbeat、Auditbeat、Heartbeat、Functionbeat。

流程

摘自网络

环境及版本

  • CentOS 7
  • jdk1.8.0
  • ELK Stack 7.0

官网地址 https://www.elastic.co/cn/elastic-stack
以下操作，都以非 root 用户操作。默认不能使用root用户。

Elasticsearch

  • 下载elasticsearch-7.7.1-linux-x86_64.tar.gz 
#解压 elasticsearch
[vagrant@localhost ~]$ tar -xvf elasticsearch-7.7.1-linux-x86_64.tar.gz 
#目录结构
[vagrant@localhost elasticsearch-7.7.1]$ ls
bin  config  jdk  lib  LICENSE.txt  logs  modules  NOTICE.txt  plugins  README.asciidoc
#配置文件目录
[vagrant@localhost elasticsearch-7.7.1]$ cd config/
[vagrant@localhost config]$ ls
elasticsearch.yml  jvm.options  jvm.options.d  log4j2.properties  role_mapping.yml  roles.yml  users  users_roles
#修改配置文件
[vagrant@localhost config]$ vi elasticsearch.yml
  • 配置 jvm.options 中的内存
[vagrant@localhost config]$ vi jvm.options
#仅需修改-Xms 和 -Xmx 
# Xms represents the initial size of total heap space
# Xmx represents the maximum size of total heap space
-Xms512m
-Xmx512m
  • 启动Elasticsearch 
# -d 后台启动 
[vagrant@localhost elasticsearch-7.7.1]$ ./bin/elasticsearch -d
future versions of Elasticsearch will require Java 11; your Java version from [/usr/java/jdk1.8.0_211-amd64/jre] does not meet this requirement
future versions of Elasticsearch will require Java 11; your Java version from [/usr/java/jdk1.8.0_211-amd64/jre] does not meet this requirement
#查看启动日志 
[vagrant@localhost elasticsearch-7.7.1]$ view logs/my-application.log 
[2020-06-18T17:04:37,992][INFO ][o.e.n.Node               ] [node-1] starting ...
[2020-06-18T17:04:38,308][INFO ][o.e.t.TransportService   ] [node-1] publish_address {192.168.0.140:9300}, bound_addresses {[::]:9300}
[2020-06-18T17:04:39,314][INFO ][o.e.b.BootstrapChecks    ] [node-1] bound or publishing to a non-loopback address, enforcing bootstrap checks
[2020-06-18T17:04:39,373][INFO ][o.e.c.c.Coordinator      ] [node-1] cluster UUID [KPfQ2tYyQeOGtgMLcGnlhA]
[2020-06-18T17:04:39,906][INFO ][o.e.c.s.MasterService    ] [node-1] elected-as-master ([1] nodes joined)[{node-1}{_TNGd5GqQ1uEeu5N-h5rWg}{JODG8Ie1TlasxO8SaJZdYA}{192.168.0.140}{192.168.0.140:9300}{dilmrt}{ml.machine_memory=1927471104, xpack.installed=true, transform.node=true, ml.max_open_jobs=20} elect leader, _BECOME_MASTER_TASK_, _FINISH_ELECTION_], term: 4, version: 48, delta: master node changed {previous [], current [{node-1}{_TNGd5GqQ1uEeu5N-h5rWg}{JODG8Ie1TlasxO8SaJZdYA}{192.168.0.140}{192.168.0.140:9300}{dilmrt}{ml.machine_memory=1927471104, xpack.installed=true, transform.node=true, ml.max_open_jobs=20}]}
[2020-06-18T17:04:40,149][INFO ][o.e.c.s.ClusterApplierService] [node-1] master node changed {previous [], current [{node-1}{_TNGd5GqQ1uEeu5N-h5rWg}{JODG8Ie1TlasxO8SaJZdYA}{192.168.0.140}{192.168.0.140:9300}{dilmrt}{ml.machine_memory=1927471104, xpack.installed=true, transform.node=true, ml.max_open_jobs=20}]}, term: 4, version: 48, reason: Publication{term=4, version=48}
[2020-06-18T17:04:40,460][INFO ][o.e.h.AbstractHttpServerTransport] [node-1] publish_address {192.168.0.140:9200}, bound_addresses {[::]:9200}
[2020-06-18T17:04:40,461][INFO ][o.e.n.Node               ] [node-1] started
[2020-06-18T17:04:41,535][INFO ][o.e.l.LicenseService     ] [node-1] license [a7d9f2e4-e934-4fa9-b11c-ada9ddd761cb] mode [basic] - valid
[2020-06-18T17:04:41,548][INFO ][o.e.x.s.s.SecurityStatusChangeListener] [node-1] Active license is now [BASIC]; Security is disabled
[2020-06-18T17:04:41,576][INFO ][o.e.g.GatewayService     ] [node-1] recovered [2] indices into cluster_state
[2020-06-18T17:04:43,550][INFO ][o.e.c.r.a.AllocationService] [node-1] Cluster health status changed from [RED] to [YELLOW] (reason: [shards started [[logstash-test-2020.06.18][0]]]).
#检查是否正常访问
[vagrant@localhost elasticsearch-7.7.1]$ curl http://192.168.0.140:9200
{
  "name" : "node-1",
  "cluster_name" : "my-application",
  "cluster_uuid" : "KPfQ2tYyQeOGtgMLcGnlhA",
  "version" : {
    "number" : "7.7.1",
    "build_flavor" : "default",
    "build_type" : "tar",
    "build_hash" : "ad56dce891c901a492bb1ee393f12dfff473a423",
    "build_date" : "2020-05-28T16:30:01.040088Z",
    "build_snapshot" : false,
    "lucene_version" : "8.5.1",
    "minimum_wire_compatibility_version" : "6.8.0",
    "minimum_index_compatibility_version" : "6.0.0-beta1"
  },
  "tagline" : "You Know, for Search"
}
  • Elasticsearch启动成功后，Kibana和Logstash启动没有先后顺序。

Kibana

  • 下载 kibana-7.7.1-linux-x86_64.tar.gz 
#解压 kibana
[vagrant@localhost ~]$ tar -xvf kibana-7.7.1-linux-x86_64.tar.gz 
#目录结构
[vagrant@localhost kibana-7.7.1]$ ls
bin  built_assets  config  data  LICENSE.txt  node  node_modules  nohup.out  NOTICE.txt  optimize  package.json  plugins  README.txt  src  webpackShims  x-pack
#修改配置文件
[vagrant@localhost kibana-7.7.1]$ vi config/kibana.yml
  • 启动 Kibana 
#后台启动，记录nohup 日志
[vagrant@localhost kibana-7.7.1]$ nohup ./bin/kibana &
[1] 18721
[vagrant@localhost kibana-7.7.1]$ nohup: ignoring input and appending output to ‘nohup.out’
#查看nohup.out日志
[vagrant@localhost kibana-7.7.1]$ tail -f -n10 nohup.out 
{"type":"log","@timestamp":"2020-06-18T17:35:41Z","tags":["warning","reporting"],"pid":18721,"message":"Enabling the Chromium sandbox provides an additional layer of protection."}
{"type":"log","@timestamp":"2020-06-18T17:35:50Z","tags":["warning","reporting"],"pid":18721,"message":"正在为 xpack.reporting.encryptionKey 生成随机密钥。要防止待处理报告在重新启动时失败，请在 kibana.yml 中设置 xpack.reporting.encryptionKey"}
{"type":"log","@timestamp":"2020-06-18T17:35:50Z","tags":["status","plugin:reporting@7.7.1","info"],"pid":18721,"state":"green","message":"Status changed from uninitialized to green - Ready","prevState":"uninitialized","prevMsg":"uninitialized"}
{"type":"log","@timestamp":"2020-06-18T17:35:53Z","tags":["listening","info"],"pid":18721,"message":"Server running at http://0.0.0.0:5601"}
{"type":"log","@timestamp":"2020-06-18T17:35:55Z","tags":["info","http","server","Kibana"],"pid":18721,"message":"http server running at http://0.0.0.0:5601"}
{"type":"log","@timestamp":"2020-06-18T17:35:56Z","tags":["error","reporting"],"pid":18721,"message":"The Reporting plugin encountered issues launching Chromium in a self-test. You may have trouble generating reports."}
{"type":"log","@timestamp":"2020-06-18T17:35:56Z","tags":["error","reporting"],"pid":18721,"message":"ErrorEvent {\n  target:\n   WebSocket {\n     _events:\n      [Object: null prototype] { open: [Function], error: [Function] },\n     _eventsCount: 2,\n     _maxListeners: undefined,\n     readyState: 3,\n     protocol: '',\n     _binaryType: 'nodebuffer',\n     _closeFrameReceived: false,\n     _closeFrameSent: false,\n     _closeMessage: '',\n     _closeTimer: null,\n     _closeCode: 1006,\n     _extensions: {},\n     _receiver: null,\n     _sender: null,\n     _socket: null,\n     _isServer: false,\n     _redirects: 0,\n     url:\n      'ws://127.0.0.1:41039/devtools/browser/fefdb355-c461-4f5c-8e1f-cfb58978cf0f',\n     _req: null },\n  type: 'error',\n  message: 'connect ECONNREFUSED 127.0.0.1:41039',\n  error:\n   { Error: connect ECONNREFUSED 127.0.0.1:41039\n       at TCPConnectWrap.afterConnect [as oncomplete] (net.js:1107:14)\n     errno: 'ECONNREFUSED',\n     code: 'ECONNREFUSED',\n     syscall: 'connect',\n     address: '127.0.0.1',\n     port: 41039 } }"}
{"type":"log","@timestamp":"2020-06-18T17:35:56Z","tags":["warning","reporting"],"pid":18721,"message":"See Chromium's log output at \"/home/vagrant/kibana-7.7.1/data/headless_shell-linux/chrome_debug.log\""}
{"type":"log","@timestamp":"2020-06-18T17:35:56Z","tags":["error","reporting"],"pid":18721,"message":"Error: Could not close browser client handle!\n    at browserFactory.test.then.browser (/home/vagrant/kibana-7.7.1/x-pack/legacy/plugins/reporting/server/lib/validate/validate_browser.js:26:15)\n    at process._tickCallback (internal/process/next_tick.js:68:7)"}
{"type":"log","@timestamp":"2020-06-18T17:35:56Z","tags":["warning","reporting"],"pid":18721,"message":"Reporting 插件自检生成警告：Error: Could not close browser client handle!"}
#查看启动状态
[vagrant@localhost kibana-7.7.1]$ netstat -anp|grep 5601
tcp        0      0 0.0.0.0:5601            0.0.0.0:*               LISTEN      18721/./bin/../node
  • 访问Kibana，http://192.168.0.140:5601
    Kibana主页，配置文件：i18n.locale: "zh-CN" 设置语言
  • Kibana启动成功

Logstash

  • 下载 logstash-7.7.1.tar.gz 
#解压 logstash
[vagrant@localhost ~]$ tar -xvf logstash-7.7.1.tar.gz
#目录结构 
[vagrant@localhost logstash-7.7.1]$ ls
bin  config  CONTRIBUTORS  data  Gemfile  Gemfile.lock  lib  LICENSE.txt  logs  logstash-core  logstash-core-plugin-api  modules  nohup.out  NOTICE.TXT  plugin-data  tools  vendor  x-pack
#配置文件目录
[vagrant@localhost config]$ ls
filebeat.conf  jvm.options  log4j2.properties  logstash.conf  logstash-sample.conf  logstash.yml  pipelines.yml  startup.options
  • 配置 jvm.options 中的内存
[vagrant@localhost config]$ vi jvm.options 
#仅需修改-Xms 和 -Xmx 
# Xms represents the initial size of total heap space
# Xmx represents the maximum size of total heap space
-Xms512m
-Xmx512m
  • 配置logstash-sample.conf文件，Logstash的事件处理分为3个阶段: inputs、filters、outputs。
  • Logstash运行时可以指定conf文件，为了介绍Filebeat的使用，创建filebeat-1.conf。
  • logstash-sample.conf配置方式很多，需要根据实际场景进行相关配置，这里只为演示，后续会单独写一篇文章介绍配置规则。
#创建 filebeat.conf 并且启动时指定
[vagrant@localhost config]$ vi filebeat.conf
#配置端口
input {
  beats {
       port => 5044
       codec => json{
            charset => "UTF-8"
        }
  }
}
#输出至elasticsearch
output {
  elasticsearch{
    hosts => ["192.168.0.140:9200"]
    #索引格式
    index => "logstash-test-%{+YYYY.MM.dd}"
  }
  stdout { codec => rubydebug }
}
  • 启动 Logstash 
# logstash本身存在日志，使用后台启动并且，不输出控制台日志
[vagrant@localhost logstash-7.7.1]$ ./bin/logstash -f ./config/filebeat.conf >/dev/null 2>&1 &
[11] 22292
#查看启动日志
[vagrant@localhost logs]$ tail -f -n10 logstash-plain.log 
[2020-06-18T18:39:31,182][INFO ][logstash.outputs.elasticsearch][main] New Elasticsearch output {:class=>"LogStash::Outputs::ElasticSearch", :hosts=>["//192.168.0.140:9200"]}
[2020-06-18T18:39:31,320][INFO ][logstash.outputs.elasticsearch][main] Using default mapping template
[2020-06-18T18:39:31,455][WARN ][org.logstash.instrument.metrics.gauge.LazyDelegatingGauge][main] A gauge metric of an unknown type (org.jruby.specialized.RubyArrayOneObject) has been created for key: cluster_uuids. This may result in invalid serialization.  It is recommended to log an issue to the responsible developer/development team.
[2020-06-18T18:39:31,470][INFO ][logstash.javapipeline    ][main] Starting pipeline {:pipeline_id=>"main", "pipeline.workers"=>1, "pipeline.batch.size"=>125, "pipeline.batch.delay"=>50, "pipeline.max_inflight"=>125, "pipeline.sources"=>["/home/vagrant/logstash-7.7.1/config/filebeat.conf"], :thread=>"#<Thread:0x1d59562d run>"}
[2020-06-18T18:39:31,567][INFO ][logstash.outputs.elasticsearch][main] Attempting to install template {:manage_template=>{"index_patterns"=>"logstash-*", "version"=>60001, "settings"=>{"index.refresh_interval"=>"5s", "number_of_shards"=>1}, "mappings"=>{"dynamic_templates"=>[{"message_field"=>{"path_match"=>"message", "match_mapping_type"=>"string", "mapping"=>{"type"=>"text", "norms"=>false}}}, {"string_fields"=>{"match"=>"*", "match_mapping_type"=>"string", "mapping"=>{"type"=>"text", "norms"=>false, "fields"=>{"keyword"=>{"type"=>"keyword", "ignore_above"=>256}}}}}], "properties"=>{"@timestamp"=>{"type"=>"date"}, "@version"=>{"type"=>"keyword"}, "geoip"=>{"dynamic"=>true, "properties"=>{"ip"=>{"type"=>"ip"}, "location"=>{"type"=>"geo_point"}, "latitude"=>{"type"=>"half_float"}, "longitude"=>{"type"=>"half_float"}}}}}}}
[2020-06-18T18:39:33,410][INFO ][logstash.inputs.beats    ][main] Beats inputs: Starting input listener {:address=>"0.0.0.0:5044"}
[2020-06-18T18:39:33,457][INFO ][logstash.javapipeline    ][main] Pipeline started {"pipeline.id"=>"main"}
[2020-06-18T18:39:33,567][INFO ][logstash.agent           ] Pipelines running {:count=>1, :running_pipelines=>[:main], :non_running_pipelines=>[]}
[2020-06-18T18:39:33,727][INFO ][org.logstash.beats.Server][main][5dc1e5946dbbc3654c409d7a6be51e5de6ae06dcf07ca7a0a3b518759207b8a6] Starting server on port: 5044
[2020-06-18T18:39:34,507][INFO ][logstash.agent           ] Successfully started Logstash API endpoint {:port=>9600}
  • Logstash启动成功，

Filebeats

  • 下载filebeat-7.7.1-linux-x86_64.tar.gz 
#解压filebeat
[vagrant@localhost ~]$ tar -xvf filebeat-7.7.1-linux-x86_64.tar.gz
#目录结构
[vagrant@localhost filebeat-7.7.1]$ ls
data  fields.yml  filebeat  filebeat.log  filebeat.reference.yml  filebeat.yml  kibana  LICENSE.txt  logs  module  modules.d  nohup.out  NOTICE.txt  README.md
  • 启动 Filebeat 
#nohup 后台启动
[vagrant@localhost filebeat-7.7.1]$ nohup ./filebeat -e -c filebeat.yml &
[5] 20426
#查看启动nohup日志
[vagrant@localhost filebeat-7.7.1]$ tail -f -n10 nohup.out 
2020-06-18T19:09:34.256Z    INFO    instance/beat.go:297    Setup Beat: filebeat; Version: 7.7.1
2020-06-18T19:09:34.256Z    INFO    [publisher] pipeline/module.go:110  Beat name: localhost.localdomain
2020-06-18T19:09:34.291Z    WARN    beater/filebeat.go:152  Filebeat is unable to load the Ingest Node pipelines for the configured modules because the Elasticsearch output is not configured/enabled. If you have already loaded the Ingest Node pipelines or are using Logstash pipelines, you can ignore this warning.
2020-06-18T19:09:34.291Z    INFO    instance/beat.go:438    filebeat start running.
2020-06-18T19:09:34.292Z    WARN    beater/filebeat.go:335  Filebeat is unable to load the Ingest Node pipelines for the configured modules because the Elasticsearch output is not configured/enabled. If you have already loaded the Ingest Node pipelines or are using Logstash pipelines, you can ignore this warning.
2020-06-18T19:09:34.292Z    INFO    registrar/registrar.go:145  Loading registrar data from /home/vagrant/filebeat-7.7.1/data/registry/filebeat/data.json
2020-06-18T19:09:34.292Z    INFO    registrar/registrar.go:152  States Loaded from registrar: 12
2020-06-18T19:09:34.292Z    INFO    beater/crawler.go:73    Loading Inputs: 1
2020-06-18T19:09:34.292Z    INFO    [monitoring]    log/log.go:118  Starting metrics logging every 30s
2020-06-18T19:09:34.293Z    INFO    log/input.go:152    Configured paths: [/home/vagrant/apache-tomcat-9.0.20/logs/catalina.*.out]
2020-06-18T19:09:34.293Z    INFO    input/input.go:114  Starting input of type: log; ID: 7647873084758492354 
2020-06-18T19:09:34.295Z    INFO    beater/crawler.go:105   Loading and starting Inputs completed. Enabled inputs: 1
2020-06-18T19:09:34.295Z    INFO    cfgfile/reload.go:175   Config reloader started
2020-06-18T19:09:34.295Z    INFO    cfgfile/reload.go:235   Loading of config files completed.
  • Filebeat启动成功，Tomcat的内容这里不赘述，直接启动然后查看catalina.out日志的同步情况
[vagrant@localhost apache-tomcat-9.0.20]$ ./bin/startup.sh
Using CATALINA_BASE:   /home/vagrant/apache-tomcat-9.0.20
Using CATALINA_HOME:   /home/vagrant/apache-tomcat-9.0.20
Using CATALINA_TMPDIR: /home/vagrant/apache-tomcat-9.0.20/temp
Using JRE_HOME:        /usr/java/jdk1.8.0_211-amd64
Using CLASSPATH:       /home/vagrant/apache-tomcat-9.0.20/bin/bootstrap.jar:/home/vagrant/apache-tomcat-9.0.20/bin/tomcat-juli.jar
Tomcat started.
  • Filebeat日志打印内容
2020-06-18T19:16:24.325Z    INFO    log/harvester.go:297    Harvester started for file: /home/vagrant/apache-tomcat-9.0.20/logs/catalina.2020-06-18.out
2020-06-18T19:16:32.331Z    INFO    [publisher_pipeline_output] pipeline/output.go:101  Connecting to backoff(async(tcp://192.168.0.140:5044))
2020-06-18T19:16:32.332Z    INFO    [publisher_pipeline_output] pipeline/output.go:111  Connection to backoff(async(tcp://192.168.0.140:5044)) established
2020-06-18T19:16:34.295Z    INFO    [monitoring]    log/log.go:145  Non-zero metrics in the last 30s    {"monitoring": {"metrics": {"beat":{"cpu":{"system":{"ticks":130,"time":{"ms":7}},"total":{"ticks":200,"time":{"ms":9},"value":200},"user":{"ticks":70,"time":{"ms":2}}},"handles":{"limit":{"hard":4096,"soft":1024},"open":11},"info":{"ephemeral_id":"06132633-b464-490e-afbc-1a6492cae22f","uptime":{"ms":420136}},"memstats":{"gc_next":9997632,"memory_alloc":7814672,"memory_total":21107344,"rss":774144},"runtime":{"goroutines":30}},"filebeat":{"events":{"added":2,"done":2},"harvester":{"files":{"72f618d7-3a12-4518-b707-414633178d64":{"last_event_published_time":"2020-06-18T19:16:31.331Z","last_event_timestamp":"2020-06-18T19:16:24.325Z","name":"/home/vagrant/apache-tomcat-9.0.20/logs/catalina.2020-06-18.out","read_offset":7192,"size":5058,"start_time":"2020-06-18T19:16:24.325Z"}},"open_files":1,"running":1,"started":1}},"libbeat":{"config":{"module":{"running":0}},"output":{"events":{"acked":1,"batches":1,"total":1},"read":{"bytes":6},"write":{"bytes":1863}},"pipeline":{"clients":1,"events":{"active":0,"filtered":1,"published":1,"retry":1,"total":2},"queue":{"acked":1}}},"registrar":{"states":{"current":10,"update":2},"writes":{"success":2,"total":2}},"system":{"load":{"1":0.07,"15":0.58,"5":0.46,"norm":{"1":0.07,"15":0.58,"5":0.46}}}}}}

  • 此时Logstash的日志中也会有信息输出

  • 打开Kibana页面，Management菜单创建索引模式

    image.png

  • Discover菜单中查看同步的日志信息

    image.png

  • 日志菜单中设置添加日志索引

    image.png

  • 流式传输中查看日志

    image.png

结尾

  • 实际使用中，会有很多详细的配置，会放在后续文章中进行讲解。
  • 还可以将MySQL的慢查询日志、MySQL错误日志、Nginx的日志，整合建立不同索引，方便我们进行查询。
最后编辑于 ：
©著作权归作者所有,转载或内容合作请联系作者
  • 人面猴
    序言：七十年代末，一起剥皮案震惊了整个滨河市，随后出现的几起案子，更是在滨河造成了极大的恐慌，老刑警刘岩，带你破解...
    沈念sama阅读 194,761评论 5赞 460
  • 死咒
    序言：滨河连续发生了三起死亡事件，死亡现场离奇诡异，居然都是意外死亡，警方通过查阅死者的电脑和手机，发现死者居然都...
    沈念sama阅读 81,953评论 2赞 371
  • 救了他两次的神仙让他今天三更去死
    文/潘晓璐 我一进店门，熙熙楼的掌柜王于贵愁眉苦脸地迎上来，“玉大人，你说我怎么就摊上这事。” “怎么了？”我有些...
    开封第一讲书人阅读 141,998评论 0赞 320
  • 道士缉凶录：失踪的卖姜人
    文/不坏的土叔 我叫张陵，是天一观的道长。 经常有香客问我，道长，这世上最难降的妖魔是什么？ 我笑而不...
    开封第一讲书人阅读 52,248评论 1赞 263
  • 港岛之恋（遗憾婚礼）
    正文 为了忘掉前任，我火速办了婚礼，结果婚礼上，老公的妹妹穿的比我还像新娘。我一直安慰自己，他们只是感情好，可当我...
    茶点故事阅读 61,130评论 4赞 356
  • 恶毒庶女顶嫁案：这布局不是一般人想出来的
    文/花漫 我一把揭开白布。 她就那样静静地躺着，像睡着了一般。 火红的嫁衣衬着肌肤如雪。 梳的纹丝不乱的头发上，一...
    开封第一讲书人阅读 46,145评论 1赞 272
  • 城市分裂传说
    那天，我揣着相机与录音，去河边找鬼。 笑死，一个胖子当着我的面吹牛，可吹牛的内容都是我干的。 我是一名探鬼主播，决...
    沈念sama阅读 36,550评论 3赞 381
  • 双鸳鸯连环套：你想象不到人心有多黑
    文/苍兰香墨 我猛地睁开眼，长吁一口气：“原来是场噩梦啊……” “哼！你这毒妇竟也来了？” 一声冷哼从身侧响起，我...
    开封第一讲书人阅读 35,236评论 0赞 253
  • 万荣杀人案实录
    序言：老挝万荣一对情侣失踪，失踪者是张志新（化名）和其女友刘颖，没想到半个月后，有当地人在树林里发现了一具尸体，经...
    沈念sama阅读 39,510评论 1赞 291
  • 护林员之死
    正文 独居荒郊野岭守林人离奇死亡，尸身上长有42处带血的脓包…… 初始之章·张勋 以下内容为张勋视角 年9月15日...
    茶点故事阅读 34,601评论 2赞 310
  • 白月光启示录
    正文 我和宋清朗相恋三年，在试婚纱的时候发现自己被绿了。 大学时的朋友给我发了我未婚夫和他白月光在一起吃饭的照片。...
    茶点故事阅读 36,376评论 1赞 326
  • 活死人
    序言：一个原本活蹦乱跳的男人离奇死亡，死状恐怖，灵堂内的尸体忽然破棺而出，到底是诈尸还是另有隐情，我是刑警宁泽，带...
    沈念sama阅读 32,247评论 3赞 313
  • 日本核电站爆炸内幕
    正文 年R本政府宣布，位于F岛的核电站，受9级特大地震影响，放射性物质发生泄漏。R本人自食恶果不足惜，却给世界环境...
    茶点故事阅读 37,613评论 3赞 299
  • 男人毒药：我在死后第九天来索命
    文/蒙蒙 一、第九天 我趴在偏房一处隐蔽的房顶上张望。 院中可真热闹，春花似锦、人声如沸。这庄子的主人今日做“春日...
    开封第一讲书人阅读 28,911评论 0赞 17
  • 一桩弑父案，背后竟有这般阴谋
    文/苍兰香墨 我抬头看了看天上的太阳。三九已至，却和暖如春，着一层夹袄步出监牢的瞬间，已是汗流浃背。 一阵脚步声响...
    开封第一讲书人阅读 30,191评论 1赞 250
  • 情欲美人皮
    我被黑心中介骗来泰国打工， 没想到刚下飞机就差点儿被人妖公主榨干…… 1. 我叫王不留，地道东北人。 一个月前我还...
    沈念sama阅读 41,532评论 2赞 342
  • 代替公主和亲
    正文 我出身青楼，却偏偏与公主长得像，于是被迫代替她去往敌国和亲。 传闻我的和亲对象是个残疾皇子，可洞房花烛夜当晚...
    茶点故事阅读 40,739评论 2赞 335