Formed in 2009, the Archive Team (not to be confused with the archive.org Archive-It Team) is a rogue archivist collective dedicated to saving copies of rapidly dying or deleted websites for the sake of history and digital heritage. The group is 100% composed of volunteers and interested parties, and has expanded into a large amount of related projects for saving online and digital history.
ecs-logging-python
Please note that this is library is in a beta version and backwards-incompatible changes might be introduced in future releases. While we strive to comply to semver, we can not guarantee to avoid breaking changes in minor releases.
Check out the Elastic Common Schema (ECS) reference for more information.
The library currently implements ECS 1.5, after a 1.x version is released we will be following (ECS.major).(ECS.minor).(package minor) as our versioning scheme.
Installation
$ python -m pip install ecs-loggingGetting Started
ecs-logging-python has formatters for the standard library
logging module
and the structlog package.
Standard Library logging Module
import logging
import ecs_logging
# Get the Logger
logger = logging.getLogger("app")
logger.setLevel(logging.DEBUG)
# Add an ECS formatter to the Handler
handler = logging.StreamHandler()
handler.setFormatter(ecs_logging.StdlibFormatter())
logger.addHandler(handler)
# Emit a log!
logger.debug("Example message!", extra={"http.request.method": "get"}){
"@timestamp": "2020-03-20T18:11:37.895Z",
"ecs": {
"version": "1.5.0"
},
"http": {
"request": {
"method": "get"
}
},
"log": {
"level": "debug",
"logger": "app",
"origin": {
"file": {
"line": 14,
"name": "test.py"
},
"function": "func"
},
"original": "Example message!"
},
"message": "Example message!"
}Excluding Fields
You can exclude fields from being collected by using the exclude_fields option
in the StdlibFormatter constructor:
from ecs_logging import StdlibFormatter
formatter = StdlibFormatter(
exclude_fields=[
# You can specify individual fields to ignore:
"log.original",
# or you can also use prefixes to ignore
# whole categories of fields:
"process",
"log.origin",
]
)Limiting Stack Traces
The StdlibLogger automatically gathers exc_info into ECS error.* fields.
If you'd like to control the number of stack frames that are included
in error.stack_trace you can use the stack_trace_limit parameter
(by default all frames are collected):
from ecs_logging import StdlibFormatter
formatter = StdlibFormatter(
# Only collects 3 stack frames
stack_trace_limit=3,
)
formatter = StdlibFormatter(
# Disable stack trace collection
stack_trace_limit=0,
)Structlog Example
import structlog
import ecs_logging
# Configure Structlog
structlog.configure(
processors=[ecs_logging.StructlogFormatter()],
wrapper_class=structlog.BoundLogger,
context_class=dict,
logger_factory=structlog.PrintLoggerFactory(),
)
# Get the Logger
logger = structlog.get_logger("app")
# Add additional context
logger = logger.bind(**{
"http": {
"version": "2",
"request": {
"method": "get",
"bytes": 1337,
},
},
"url": {
"domain": "example.com",
"path": "/",
"port": 443,
"scheme": "https",
"registered_domain": "example.com",
"top_level_domain": "com",
"original": "https://example.com",
}
})
# Emit a log!
logger.debug("Example message!"){
"@timestamp": "2020-03-26T13:08:11.728Z",
"ecs": {
"version": "1.5.0"
},
"http": {
"request": {
"bytes": 1337,
"method": "get"
},
"version": "2"
},
"log": {
"level": "debug"
},
"message": "Example message!",
"url": {
"domain": "example.com",
"original": "https://example.com",
"path": "/",
"port": 443,
"registered_domain": "example.com",
"scheme": "https",
"top_level_domain": "com"
}
}Elastic APM Log Correlation
ecs-logging-python supports automatically collecting ECS tracing fields
from the Elastic APM Python agent in order to
correlate logs to spans, transactions and traces in Elastic APM.
License
Apache-2.0
