Apache NiFi 2.0.0-M2 Out!
https://medium.com/@tspann/apache-nifi-2-0-0-m2-out-314a1d4c8b20
Apache NiFi 2.0.0-M2 Out!
New NiFi Features and Updates
More NiFi and faster.
So where’s the beef in this new upgrade?
New Features and Changes
New Schema Registry Options Added
- DatabaseTableSchemaRegistryService
- StandardJsonSchemaRegistry
New Components
- StandardKustoIngestService
- ZendeskRecordSink
New Processors
- CalculateParquetOffsets
- CalculateParquetRowGroupOffsets
- FilterAttribute
- PublishSlack
- PutMongoBulk
- PutAzureDataExplorer
- PutZendeskTicket
NiFi Used Libraries / Upgraded
- Spring Framework 6
- Jetty 12
- Jakarta Servlet API 6
- Jakarta XML Binding 4
- Swagger 2 annotations
- OpenAPI 3.0 REST API specification
These removals may hurt
- Removed MiNiFi C2 Server modules
- Removed Docker image configuration
- Relocated JoltTransformJSON and JoltTransformRecord from nifi-standard-nar to nifi-jolt-nar
- Removed InfluxDB Processors (use processors from an older version if needed)
- Removed Bootstrap Notification Services
ListenSlack (WebSockets API)
Now we can use this processor (and not need to leave a port open for Slack to call us), to get the current stream of Slack messages. This is great and fast. WebSockets are nice and not having to write this myself is nice.
{
"clientMsgId" : "3434ad0d-0afe-4563-8c21-91bda87cf41c",
"type" : "message",
"team" : "E2TE1MAG",
"channel" : "C1SD6N197",
"user" : "ULMRENSE4",
"botId" : null,
"botProfile" : null,
"text" : "Q: What is the weather at Newark?",
"blocks" : [ {
"type" : "rich_text",
"elements" : [ {
"type" : "rich_text_section",
"elements" : [ {
"type" : "text",
"text" : "Q: What is the weather at Newark?",
"style" : null
} ]
} ],
"blockId" : "z/mKt"
} ],
"attachments" : null,
"files" : null,
"ts" : "1706648003.547529",
"parentUserId" : null,
"threadTs" : null,
"eventTs" : "1706648003.547529",
"channelType" : "channel",
"edited" : null,
"subtype" : null
}
ConsumeSlack
For Consume Slack we are able to grab the history from Slack, which is great.
[{"type":"message","subtype":null,"team":"A7TE32HJKA",
"channel":"C1SD6N197","user":"ULMS1759T","username":null,
"text":"Q: When did Emirates Airlines start?",
"blocks":[{"type":"rich_text",
"elements":[{"type":"rich_text_section",
"elements":[{"type":"text","text":
"Q: When did Emirates Airlines start?","style":null}]}],
"blockId":"bTFip"}],"attachments":null, ...
PublishSlack
slack.channel.id
C05QAAVEC0H
slack.ts
1706642875.023669This new one is nice as you just pass in a FlowFile, but warning. PutSlack is gone!!! No more incoming webhooks used. You need an Access Token from Slack.
DatabaseTableSchemaRegistry
FilterAttribute
NiFi 1.25
For a production branch, NiFi 1.x has been upgraded and has some goodies. It has the Slack, FilterAttributes and DatabaseSchemaRegistry. So this is where you should be running your main production flows (or preferrably in Cloudera DataFlow with full support).
