Introduction - eMagiz

We will elaborate on these concepts a bit more so you have a good introduction before we start linking Event Streaming to eMagiz in the upcoming microlearnings.

22

23

== 3. Event Streaming - Introduction ==

24

25

In this microlearning, we want to take you on a short journey to explain the fundamental concepts of what Event Streaming is.

26

In the microlearnings that follow we will guide you in implementing event streaming within the eMagiz platform.

27

28

We will explain what Event Streaming is based on the following perspectives:

29

30

* With event streaming, you can publish data without building specific data integration between one or more applications

31

* Publisher and subscriber are de-coupled and don't need to be aware of each other specifically

32

* Data is stored inside the platform for a certain amount of time (retention)

33

* Event data is continuous being sent or made available

34

35

=== 3.1 Publish data without building a data integration ===

36

37

Event Streaming revolves around a cluster of topics that contain data. External parties can write data to said topics without having to know who will consume the data.

38

Compare it to a news item that gets published on a news website. The editor publishes the data on the news site without

39

knowing who will consume the data and when they will consume the data.

40

41

The same principle applies to other types of data. By having a centralized point where authorized parties can publish their data for others to consume you have enough to exchange that data between parties.

42

Subsequently, it is up to the consuming entities to read the data and process the data in a way that aids their business process.

43

44

[[image:Main.Images.Microlearning.WebHome@crashcourse-eventstreaming-event-streaming-introduction--centralized-data-set.png]]

45

46

=== 3.2 Publisher and subscriber are de-coupled ===

47

48

Because there is a centralized managed cluster of topics to which a producer can write data without having to wait for acknowledgment of the consumer(s)

49

you can de-couple all publishers and subscribers. This makes sure that your microservices landscape can be strongly decoupled.

50

51

By being de-coupled from one another it makes it easier to develop on certain applications in parallel without having to constantly communicate on the shared integration that transfers data from application A to application B.

52

53

In this way, it has become the sole responsibility of the producer to publish data on a certain topic. After he has done so his job is finished.

54

55

It is further up to the consumer to subscribe to that data and process it in a way that adds value to their specific business application.

56

57

[[image:Main.Images.Microlearning.WebHome@crashcourse-eventstreaming-event-streaming-introduction--de-coupling-publish-subscribe.jpg]]

58

59

=== 3.3 Data is retained inside the platform ===

60

61

Because publishers and subscribers are unaware of each other a centralized piece

62

of the puzzle needs to make sure that data is kept at that centralized location (the topic) for a certain amount of time.

63

64

This way you can ensure that data can be consumed at will by those who subscribed to a topic without having to navigate to the

65

transactional nature of delivering a specific message to each consumer as you would with messaging.

66

67

Correctly determining the retention policy in terms of event streaming is, therefore, one of the crucial parts of a successful implementation of event streaming.

68

69

[[image:Main.Images.Microlearning.WebHome@crashcourse-eventstreaming-event-streaming-introduction--retention-construction.jpg]]

70

71

=== 3.4 Event data is continuous in nature ===

72

73

The 'streaming' part of the event stream says it all. In this scenario, the data (compiled of a continuous series of events) keeps on being sent to the topic and is therefore continuously available for consumers to process.

74

75

For example, if you want to monitor the temperature in a certain building on a room by room basis to send out an alert when the temperature passes a certain threshold you can facilitate this easily with Event Streaming.

76

77

Simply make sure that all monitors send their data to the topic continuously and check whether or not a value of a monitor exceeds a value.

78

If so, please the message on a different topic from which a third application can read the data and send an SMS to a concerned party.

79

80

This is a very rudimentary scenario that can be optimized a lot but it gives an idea of what Event Streaming can do for you.

81

82

[[image:Main.Images.Microlearning.WebHome@crashcourse-eventstreaming-event-streaming-introduction--log-record-event-streaming.png]]

83

84

=== 3.5 Example situations Event Streaming ===

85

86

* Sensors in vehicles, industrial equipment, and farm machinery send data to a streaming application. The application monitors performance detects any potential defects in advance and places a spare part order automatically preventing equipment downtime.

87

* A financial institution tracks changes in the stock market in real-time, computes value-at-risk, and automatically rebalances portfolios based on stock price movements.

88

* A real-estate website tracks a subset of data from consumers’ mobile devices and makes real-time property recommendations of properties to visit based on their geo-location.

89

* A solar power company has implemented a streaming data application that monitors all of the panels in the field, and schedules service in real-time, thereby minimizing the periods of low throughput from each panel and the associated penalty payouts.

90

* A media publisher streams billions of clickstream records from its online properties, aggregates and enriches the data with demographic information about users, and optimizes content placement on its site, delivering relevancy and a better experience to its audience.

91

* An online gaming company collects streaming data about player-game interactions and feeds the data into its gaming platform. It then analyzes the data in real-time, offers incentives and dynamic experiences to engage its players.

== 4. Assignment ==

For this microlearning, there is no assignment.

96

97

== 5. Key takeaways ==

98

99

Event Streaming is a decoupled service that works best within a microservices architecture with each application having its core function for high volumes.

100

101

Critical perspectives when looking at Event Streaming are:

102

103

* With event streaming, you can publish data without building specific data integration between one or more applications

104

* Publisher and subscriber are de-coupled and don't need to be aware of each other specifically

105

* Data is stored inside the platform for a certain amount of time (retention)

106

* Event data is continuous being sent or made available

107

108

== 6. Suggested Additional Readings ==

109

110

If you are interested in this topic and want more information on it please see the following links:

111

112

* https://www.emagiz.com/event-streaming/

113

* https://www.confluent.io/learn/kafka-tutorial/

114

* https://www.oreilly.com/radar/the-world-beyond-batch-streaming-101/

115

-

116

117

== 7. Silent demonstration video ==

118

119

This video gives a nice introduction to Event Streaming concerning the eMagiz platform.

120

121

[[Link to platform feature Event Streaming>>https://www.youtube.com/embed/VRGz3z_T3mw]]

122

123

)))((({{toc/}}))){{/container}}{{/container}}

version	line-number	content
32.1	1	{{container}}{{container layoutStyle="columns"}}(((
	2	In this microlearning, we will give an introduction to Event Streaming and look a little bit into how this is implemented in eMagiz.
	3
	4	Should you have any questions, please contact [[academy@emagiz.com>>mailto:academy@emagiz.com]].
	5
	6	== 1. Prerequisites ==
	7
	8	* Basic knowledge of the eMagiz platform
	9
	10	== 2. Key concepts ==
	11
	12	This microlearning gives an introduction to Event Streaming.
	13
	14	Below you find several key concepts that define (parts of) event streaming:
	15
	16	* With event streaming, you can publish data without building specific data integration between one or more applications
	17	* Publisher and subscriber are de-coupled and don't need to be aware of each other specifically
	18	* Data is stored inside the platform for a certain amount of time (retention)
	19	* Event data is continuous being sent or made available
	20
	21	We will elaborate on these concepts a bit more so you have a good introduction before we start linking Event Streaming to eMagiz in the upcoming microlearnings.
	22
	23	== 3. Event Streaming - Introduction ==
	24
	25	In this microlearning, we want to take you on a short journey to explain the fundamental concepts of what Event Streaming is.
	26	In the microlearnings that follow we will guide you in implementing event streaming within the eMagiz platform.
	27
	28	We will explain what Event Streaming is based on the following perspectives:
	29
	30	* With event streaming, you can publish data without building specific data integration between one or more applications
	31	* Publisher and subscriber are de-coupled and don't need to be aware of each other specifically
	32	* Data is stored inside the platform for a certain amount of time (retention)
	33	* Event data is continuous being sent or made available
	34
	35	=== 3.1 Publish data without building a data integration ===
	36
	37	Event Streaming revolves around a cluster of topics that contain data. External parties can write data to said topics without having to know who will consume the data.
	38	Compare it to a news item that gets published on a news website. The editor publishes the data on the news site without
	39	knowing who will consume the data and when they will consume the data.
	40
	41	The same principle applies to other types of data. By having a centralized point where authorized parties can publish their data for others to consume you have enough to exchange that data between parties.
	42	Subsequently, it is up to the consuming entities to read the data and process the data in a way that aids their business process.
	43
	44	[[image:Main.Images.Microlearning.WebHome@crashcourse-eventstreaming-event-streaming-introduction--centralized-data-set.png]]
	45
	46	=== 3.2 Publisher and subscriber are de-coupled ===
	47
	48	Because there is a centralized managed cluster of topics to which a producer can write data without having to wait for acknowledgment of the consumer(s)
	49	you can de-couple all publishers and subscribers. This makes sure that your microservices landscape can be strongly decoupled.
	50
	51	By being de-coupled from one another it makes it easier to develop on certain applications in parallel without having to constantly communicate on the shared integration that transfers data from application A to application B.
	52
	53	In this way, it has become the sole responsibility of the producer to publish data on a certain topic. After he has done so his job is finished.
	54
	55	It is further up to the consumer to subscribe to that data and process it in a way that adds value to their specific business application.
	56
	57	[[image:Main.Images.Microlearning.WebHome@crashcourse-eventstreaming-event-streaming-introduction--de-coupling-publish-subscribe.jpg]]
	58
	59	=== 3.3 Data is retained inside the platform ===
	60
	61	Because publishers and subscribers are unaware of each other a centralized piece
	62	of the puzzle needs to make sure that data is kept at that centralized location (the topic) for a certain amount of time.
	63
	64	This way you can ensure that data can be consumed at will by those who subscribed to a topic without having to navigate to the
	65	transactional nature of delivering a specific message to each consumer as you would with messaging.
	66
	67	Correctly determining the retention policy in terms of event streaming is, therefore, one of the crucial parts of a successful implementation of event streaming.
	68
	69	[[image:Main.Images.Microlearning.WebHome@crashcourse-eventstreaming-event-streaming-introduction--retention-construction.jpg]]
	70
	71	=== 3.4 Event data is continuous in nature ===
	72
	73	The 'streaming' part of the event stream says it all. In this scenario, the data (compiled of a continuous series of events) keeps on being sent to the topic and is therefore continuously available for consumers to process.
	74
	75	For example, if you want to monitor the temperature in a certain building on a room by room basis to send out an alert when the temperature passes a certain threshold you can facilitate this easily with Event Streaming.
	76
	77	Simply make sure that all monitors send their data to the topic continuously and check whether or not a value of a monitor exceeds a value.
	78	If so, please the message on a different topic from which a third application can read the data and send an SMS to a concerned party.
	79
	80	This is a very rudimentary scenario that can be optimized a lot but it gives an idea of what Event Streaming can do for you.
	81
	82	[[image:Main.Images.Microlearning.WebHome@crashcourse-eventstreaming-event-streaming-introduction--log-record-event-streaming.png]]
	83
	84	=== 3.5 Example situations Event Streaming ===
	85
	86	* Sensors in vehicles, industrial equipment, and farm machinery send data to a streaming application. The application monitors performance detects any potential defects in advance and places a spare part order automatically preventing equipment downtime.
	87	* A financial institution tracks changes in the stock market in real-time, computes value-at-risk, and automatically rebalances portfolios based on stock price movements.
	88	* A real-estate website tracks a subset of data from consumers’ mobile devices and makes real-time property recommendations of properties to visit based on their geo-location.
	89	* A solar power company has implemented a streaming data application that monitors all of the panels in the field, and schedules service in real-time, thereby minimizing the periods of low throughput from each panel and the associated penalty payouts.
	90	* A media publisher streams billions of clickstream records from its online properties, aggregates and enriches the data with demographic information about users, and optimizes content placement on its site, delivering relevancy and a better experience to its audience.
	91	* An online gaming company collects streaming data about player-game interactions and feeds the data into its gaming platform. It then analyzes the data in real-time, offers incentives and dynamic experiences to engage its players.
	92
	93	== 4. Assignment ==
	94
	95	For this microlearning, there is no assignment.
	96
	97	== 5. Key takeaways ==
	98
	99	Event Streaming is a decoupled service that works best within a microservices architecture with each application having its core function for high volumes.
	100
	101	Critical perspectives when looking at Event Streaming are:
	102
	103	* With event streaming, you can publish data without building specific data integration between one or more applications
	104	* Publisher and subscriber are de-coupled and don't need to be aware of each other specifically
	105	* Data is stored inside the platform for a certain amount of time (retention)
	106	* Event data is continuous being sent or made available
	107
	108	== 6. Suggested Additional Readings ==
	109
	110	If you are interested in this topic and want more information on it please see the following links:
	111
	112	* https://www.emagiz.com/event-streaming/
	113	* https://www.confluent.io/learn/kafka-tutorial/
	114	* https://www.oreilly.com/radar/the-world-beyond-batch-streaming-101/
	115	-
	116
	117	== 7. Silent demonstration video ==
	118
	119	This video gives a nice introduction to Event Streaming concerning the eMagiz platform.
	120
	121	[[Link to platform feature Event Streaming>>https://www.youtube.com/embed/VRGz3z_T3mw]]
	122
	123	)))((({{toc/}}))){{/container}}{{/container}}

Wiki source code of Introduction