Dell EMC Streaming Data Platform Enables Insights for Streaming Data from the Edge

Streaming Data Creates Massive Potential for Organizations Across Industries According to IDC, more than a quarter of data created in the global datasphere will be real-time in nature by 2025.[i] Much of that data will come from the edge, originate from sensors, cameras and drones, and come in the form of a continuous data stream. Streaming data creates additional complexities in the already intricate world of unstructured data. With its tendency to vary in volume and boundaries and with timestamps that can fluctuate out of order, the need for a Data First infrastructure – where organizations … READ MORE

Related:

4 Reasons Why Businesses Need Data Lakes

It’s common knowledge in the modern business world that big data —that is, the large volumes of data collected for analysis by organizations—is a significant element of any strategy—operations, sales, marketing, finance, HR, and every other department rely on big data solutions to stay ahead of the competition. But how that big data is handled remains another story.

Enter the world of data lakes. Data lakes are repositories that can take in data from multiple sources. Rather than process data for immediate analysis, all received data is stored in its native format. This model allows data lakes to hold massive amounts of data while using minimal resources. Data is only processed upon being called for usage (compared to a data warehouse, which processes all incoming data). This ultimately allows data lakes to be an efficient way for storage, resource management, and data preparation.

But do you actually need a data lake, especially if your big data solution already has a data warehouse? The answer is a resounding yes. In a world where the volume of data transmitted across countless devices continues to increase, a resource-efficient means of accessing data is critical to a successful organization. In fact, here are four specific reasons why the need for a data lake is only going to get more urgent as time goes on.

Never miss an update about big data! Subscribe to the Big Data Blog to receive the latest posts straight to your inbox!

90% of data has been generated since 2016

90% of all data ever is a lot—or is it? Consider what has become available to people as Wi-Fi, smartphones, and high-speed data networks have entered everyday life over the past twenty years. In the early 2000s, streaming was limited to audio, while broadband internet was used mostly for web surfing, emailing, and downloads. In that paradigm, device data was at a minimum and the actual data consumed was mostly about interpersonal communication, especially because videos and TV hadn’t hit a level of compression that supported high-quality streaming. Towards the end of the decade, smartphones became common and Netflix had shifted its business priority to streaming.

That means between 2010 and 2020, the internet has seen the growth of smartphones (and their apps), social media, streaming services for both audio and video, streaming video game platforms, software delivered through downloads rather than physical media, and so on, all creating exponential consumption of data. As for the part that is the most relevant to business? Consider how many businesses have associated apps constantly transmitting data to and from devices, whether to control appliances, provide instructions and specifications, or quietly transmit user metrics in the background.

With 5G data networks widely starting to deploy in 2019, bandwidths and speeds are only going to get better. This means as massive—and significant—as big data has already been in the past few years, it’s only going to get bigger as technology allows the world to become even more connected. Is your data repository ready?

95% of businesses handle unstructured data

In a digital world, businesses collect data from all types of sources, and most of that is unstructured. Consider the data collected by a company that sells services and makes appointments via an app. While some of that data comes structured—that is, in predefined formats and fields such as phone numbers, dates, transaction prices, time stamps, etc.—a company like that still has to archive and store a lot of unstructured data. Unstructured data is any type of data that doesn’t contain an inherent structure or predefined model, which makes it difficult to search, sort, and analyze without further preparation.

For the example above, unstructured data comes in a wide range of formats. For a user making an appointment, any text fields filled out to make that appointment count as unstructured data. Within the company itself, emails and documents are another form of unstructured data. The posts from a company’s social media channel are also unstructured data. Any photos or videos used by employees as notes while performing services are unstructured data. Similarly, any instructional videos or podcasts created by the company as marketing assets are also unstructured.

Unstructured data is everywhere, and as more devices connect to deliver a greater range of information, it becomes clear that organizations need a way to get their proverbial arms around all of it.

4.4 GB of data are used by Americans every minute

More than 325 million people live in the US. Nearly 70% of them have smartphones. And even if you don’t count the people currently streaming media, consider what is happening on an average smartphone in a minute. It’s receiving an update on the weather. It’s checking for any new emails in the user’s inbox. It’s pushing data to social media, delivering voicemail over Wi-Fi, delivering strategic marketing notifications from apps, such as when a real estate app pushes a new housing listing. It’s sending text and images via chat apps, and downloading app/OS updates in the background.

Data is everywhere now, which means the minute that just passed while you read the above paragraph, gigabytes of data have been transmitted across the country—4.4 million GB of data every minute, according to Domo’s Data Never Sleeps report. And that’s just the United States; when combined with the rest of the world, the total volume of data grows exponentially. For businesses, collecting this kind of data is vital to all aspects of operations, from marketing to sales to communication. Thus, every organization must put a premium on safe, available, and accessible storage.

50% of businesses say that big data has changed their sales and marketing

Most people think of big data in terms of the technical aspects. Clearly, a company that works through a phone app or provides a form of streaming uses big data and is delivering a service that simply wasn’t feasible twenty years ago. However, big data is much more than delivery of streaming content. It can create significant improvements in sales and marketing—so much so that according to a McKinsey report, 50% of businesses say that big data is driving them to change their approach in these departments.

What’s the reason for this? With big data, organizations have a much more efficient path to understanding customers than in-person focus groups. Data allows for gathering a mass sample of actions from existing and potential customers. Everything from their website browsing prior to conversion to how long they engaged with certain features of a product or service are all available at high volume, which creates a large enough sample size for a reliable customer model. To be in the cutting-edge 50%, an organization needs to have the data infrastructure to receive, store, and retrieve massive amounts of structured and unstructured data for processing.

Basically, you need a data lake

The above statistics all point to one thing—your organization needs a data lake. And if you don’t get ahead of the curve now in terms of managing data, it’s clear that the world will pass you by in all areas: operations, sales, marketing, communications, and other departments. Data is simply a way of life now, enabling precise insight-driven decisions and unparalleled discovery into root causes. When combined with machine learning and artificial intelligence, this data also allows for predictive modeling for future actions.

Learn more about why data lakes are the future of big data and discover Oracle’s big data solutions—and don’t forget to subscribe to the Oracle Big Data blog to get the latest posts sent to your inbox.

(Note: Corrected typo from Domo’s Data Never Sleeps citation.)

Related:

Event ID 352 — Unicast Streaming

Event ID 352 — Unicast Streaming

Updated: November 17, 2007

Applies To: Windows Server 2008

You can configure Unicast Streaming plug-ins in Windows Media Services to enable the distribution of content using unicast streaming, the default method by which a Windows Media server delivers content. A unicast stream is a one-to-one connection between the server and a client, which means that each client receives a distinct stream and only those clients that request the stream receive it. It offers the benefits of interactivity between a player and server, easier setup, and multiple-bit-rate (MBR) streaming capability. However, the number of users that are able to receive unicast streams is limited by content bit rate and the speed of the server network. For more information, see Delivering content as a unicast stream.

Event Details

Product: Windows Media Services
ID: 352
Source: WMServer
Version: 9.5
Symbolic Name: WMS_EVMSG_NACKS_WARNING
Message: The number of negative acknowledgements (NACKs) received by the Windows Media server exceeded the NACK warning limit. The server received %1 NACK requests in %2 milliseconds.

Resolve
Protect against denial-of-service attacks

Ordinarily, negative acknowledgement (NACK) requests occur when the server or network is overloaded and packets cannot be sent through the User Datagram Protocol (UDP) reliably. Clients must request that packets be resent. This NACK warning limit in Windows Media Services is set so that only a very high number of NACK requests, a condition that indicates a denial-of-service attack, cause this issue.

First, confirm that bandwidth bottleneck issues on the network are not causing the problem. For more information, see Bandwidth. If the network does not appear to be at fault, review the server log files to determine whether clients are instigating a denial-of-service attack by flooding the network with content requests so that the server cannot respond adequately to legitimate client requests for content. If your system is subjected to a denial-of-service attack, the log files can help you determine which clients are being used in the attack. For more information about the fields used in Windows Media server log files, see Logging Model for Windows Media Services.

Note: A streaming media network that has been correctly planned and configured will improve response time, data throughput, content availability, and reduce the data error rate. To estimate the server requirements that are necessary to ensure that your content can reach all your clients without delays or interruptions, see Capacity planning. To test the capacity of your Windows Media server, you can simulate client requests for unicast streams from the server by using Microsoft Windows Media Load Simulator.

Verify

To verify that the unicast stream can be delivered to clients, test the stream by using Windows Media Player:

  1. If you want to test the stream by using Windows Media Player on the computer that is running Windows Media Services, you must install Desktop Experience. For more information, see Installing Desktop Experience.
  2. On the Windows Media server, open Windows Media Services. To open Windows Media Services, click Start, point to Administrative Tools, and then click Windows Media Services.
  3. In the console tree, click the publishing point that hosts the stream that you want to test.
  4. In the details pane, click the Announce tab, and then, in Connect to a unicast stream, note the value of the URL that a client can use to access the content.
  5. Start Windows Media Player on a computer that can access the stream, and enter the URL that you noted in the previous step.
  6. Using the Player controls, test the control functionality of the content stream. Broadcast streams can use the Start and Stop commands. On-demand streams can use the Start, Stop, and Pause commands, and the Seek bar.
  7. Test all the available streaming protocols. A unicast stream will try to connect by using the MMS protocol, but will switch to the RTSP protocol if network conditions or the Player version requires it. The HTTP protocol is not active unless the WMS HTTP Server Control Protocol plug-in is enabled. For more information, see About data transfer protocols.
  8. Allow the stream to play for a representative period of time and check that the stream quality is sufficient for the type of content and the capabilities of the equipment.

Note: If some members of your expected audience will access the stream from outside your network firewall, your testing scenario should include that condition. For more information about the firewall configuration for Windows Media Services, see Firewall Information for Windows Media Services.

Note: To ensure that your content can reach all your clients without delays or interruptions, perform network load tests by using Microsoft Windows Media Load Simulator to determine the maximum capacity of your server, and then make the appropriate adjustments to the Limits properties in Windows Media Services that specify the Windows Media server performance boundaries. A streaming media network that has been correctly planned and configured will improve response time, data throughput, content availability, and reduce the data error rate. To estimate the server requirements that are necessary to ensure that your transmission does not exceed the capabilities of your server, network, or audience, see Capacity planning.

Related Management Information

Unicast Streaming

Streaming Media Services

Related:

Event ID 351 — Multicast Streaming

Event ID 351 — Multicast Streaming

Updated: November 17, 2007

Applies To: Windows Server 2008

You can configure Multicast Streaming plug-ins in Windows Media Services to enable the multicast distribution of content. Multicast streaming is a one-to-many relationship between a Windows Media server and the clients receiving the stream. With a multicast stream, the server streams to a multicast IP address on the network, and all clients receive the same stream by subscribing to the IP address. Because there is only one stream from the server regardless of the number of clients receiving the stream, a multicast stream requires the same amount of bandwidth as a single unicast stream containing the same content. For more information, see Delivering content as a multicast stream.

Event Details

Product: Windows Media Services
ID: 351
Source: WMServer
Version: 9.5
Symbolic Name: WMS_EVMSG_MULTICAST_FORMAT_NOT_FOUND
Message: Stream format information could not be found for the multicast broadcast.

Resolve
Create a new multicast information file

Run the Multicast Announcement Wizard for the broadcast publishing point to create a new multicast information file.

To start the Multicast Announcement Wizard:

  1. On the Windows Media server, open Windows Media Services. To open Windows Media Services, click Start, point to Administrative Tools, and then click Windows Media Services.
  2. In the console tree, right-click the broadcast publishing point for the multicast content.
  3. In the details pane, click the Announce tab.
  4. In Connect to a multicast stream, click Run Multicast Announcement Wizard.

Note: For information about completing the wizard, see Working with the Multicast Announcement Wizard.

Verify

To verify that the multicast stream can be delivered to clients, test the stream by using Windows Media Player:

  1. If you want to test the stream by using Windows Media Player on the computer that is running Windows Media Services, you must install Desktop Experience. For more information, see Installing Desktop Experience.
  2. On the Windows Media server, open Windows Media Services. To open Windows Media Services, click Start, point to Administrative Tools, and then click Windows Media Services.
  3. In the console tree, click the broadcast publishing point that hosts the stream that you want to test.
  4. In the details pane, click the Announce tab.
  5. In the Connect to a multicast stream area, click Run Multicast Announcement Wizard to create a multicast information file (a file with an .nsc file name extension). This file contains information that the Player needs to decode and stream the multicast broadcast. For more information about completing the wizard, see Working with the Multicast Announcement Wizard.
  6. Use the announcement files created by the wizard to access the multicast broadcast in Windows Media Player. For more information, see Testing the announcement file.

Related Management Information

Multicast Streaming

Streaming Media Services

Related:

Event ID 349 — Unicast Streaming

Event ID 349 — Unicast Streaming

Updated: August 14, 2009

Applies To: Windows Server 2008 R2

You can configure Unicast Streaming plug-ins in Windows Media Services to enable the distribution of content using unicast streaming, the default method by which a Windows Media server delivers content. A unicast stream is a one-to-one connection between the server and a client, which means that each client receives a distinct stream and only those clients that request the stream receive it. It offers the benefits of interactivity between a player and server, easier setup, and multiple-bit-rate (MBR) streaming capability. However, the number of users that are able to receive unicast streams is limited by content bit rate and the speed of the server network. For more information, see Delivering content as a unicast stream.

Event Details

Product: Windows Media Services
ID: 349
Source: WMServer
Version: 9.6
Symbolic Name: WMS_EVMSG_STOP_RECEIVING_NACKS
Message: The number of negative acknowledgements (NACKs) received by the Windows Media server exceeded the NACK warning limit. The server received %1 NACK requests in %2 milliseconds. The server will stop receiving NACK requests for %3 milliseconds so that it can recover.

Resolve
Protect against denial-of-service attacks

Ordinarily, negative acknowledgement (NACK) requests occur when the server or network is overloaded and packets cannot be sent through the User Datagram Protocol (UDP) reliably. Clients must request that packets be resent. This NACK warning limit in Windows Media Services is set so that only a very high number of NACK requests, a condition that indicates a denial-of-service attack, cause this issue.

First, confirm that bandwidth bottleneck issues on the network are not causing the problem. For more information, see Bandwidth. If the network does not appear to be at fault, review the server log files to determine whether clients are instigating a denial-of-service attack by flooding the network with content requests so that the server cannot respond adequately to legitimate client requests for content. If your system is subjected to a denial-of-service attack, the log files can help you determine which clients are being used in the attack. For more information about the fields used in Windows Media server log files, see Logging Model for Windows Media Services.

Note: A streaming media network that has been correctly planned and configured will improve response time, data throughput, content availability, and reduce the data error rate. To estimate the server requirements that are necessary to ensure that your content can reach all your clients without delays or interruptions, see Capacity planning. To test the capacity of your Windows Media server, you can simulate client requests for unicast streams from the server by using Microsoft Windows Media Load Simulator.

Verify

To verify that the unicast stream can be delivered to clients, test the stream by using Windows Media Player:

  1. If you want to test the stream by using Windows Media Player on the computer that is running Windows Media Services, you must install Desktop Experience. For more information, see Installing Desktop Experience.
  2. On the Windows Media server, open Windows Media Services. To open Windows Media Services, click Start, point to Administrative Tools, and then click Windows Media Services.
  3. In the console tree, click the publishing point that hosts the stream that you want to test.
  4. In the details pane, click the Announce tab, and then, in Connect to a unicast stream, note the value of the URL that a client can use to access the content.
  5. Start Windows Media Player on a computer that can access the stream, and enter the URL that you noted in the previous step.
  6. Using the Player controls, test the control functionality of the content stream. Broadcast streams can use the Start and Stop commands. On-demand streams can use the Start, Stop, and Pause commands, and the Seek bar.
  7. Test all the available streaming protocols. A unicast stream will try to connect by using the MMS protocol, but will switch to the RTSP protocol if network conditions or the Player version requires it. The HTTP protocol is not active unless the WMS HTTP Server Control Protocol plug-in is enabled. For more information, see About data transfer protocols.
  8. Allow the stream to play for a representative period of time and check that the stream quality is sufficient for the type of content and the capabilities of the equipment.

Note: If some members of your expected audience will access the stream from outside your network firewall, your testing scenario should include that condition. For more information about the firewall configuration for Windows Media Services, see Firewall Information for Windows Media Services.

Note: To ensure that your content can reach all your clients without delays or interruptions, perform network load tests by using Microsoft Windows Media Load Simulator to determine the maximum capacity of your server, and then make the appropriate adjustments to the Limits properties in Windows Media Services that specify the Windows Media server performance boundaries. A streaming media network that has been correctly planned and configured will improve response time, data throughput, content availability, and reduce the data error rate. To estimate the server requirements that are necessary to ensure that your transmission does not exceed the capabilities of your server, network, or audience, see Capacity planning.

Related Management Information

Unicast Streaming

Streaming Media Services

Related: