Author: Patrik Borosch
Publisher: Packt Publishing Ltd
ISBN: 1800562144
Category : Computers
Languages : en
Pages : 520
Get Book Here
Book Description
A practical guide to implementing a scalable and fast state-of-the-art analytical data estate Key FeaturesStore and analyze data with enterprise-grade security and auditingPerform batch, streaming, and interactive analytics to optimize your big data solutions with easeDevelop and run parallel data processing programs using real-world enterprise scenariosBook Description Azure Data Lake, the modern data warehouse architecture, and related data services on Azure enable organizations to build their own customized analytical platform to fit any analytical requirements in terms of volume, speed, and quality. This book is your guide to learning all the features and capabilities of Azure data services for storing, processing, and analyzing data (structured, unstructured, and semi-structured) of any size. You will explore key techniques for ingesting and storing data and perform batch, streaming, and interactive analytics. The book also shows you how to overcome various challenges and complexities relating to productivity and scaling. Next, you will be able to develop and run massive data workloads to perform different actions. Using a cloud-based big data-modern data warehouse-analytics setup, you will also be able to build secure, scalable data estates for enterprises. Finally, you will not only learn how to develop a data warehouse but also understand how to create enterprise-grade security and auditing big data programs. By the end of this Azure book, you will have learned how to develop a powerful and efficient analytical platform to meet enterprise needs. What you will learnImplement data governance with Azure servicesUse integrated monitoring in the Azure Portal and integrate Azure Data Lake Storage into the Azure MonitorExplore the serverless feature for ad-hoc data discovery, logical data warehousing, and data wranglingImplement networking with Synapse Analytics and Spark poolsCreate and run Spark jobs with Databricks clustersImplement streaming using Azure Functions, a serverless runtime environment on AzureExplore the predefined ML services in Azure and use them in your appWho this book is for This book is for data architects, ETL developers, or anyone who wants to get well-versed with Azure data services to implement an analytical data estate for their enterprise. The book will also appeal to data scientists and data analysts who want to explore all the capabilities of Azure data services, which can be used to store, process, and analyze any kind of data. A beginner-level understanding of data analysis and streaming will be required.
Author: Has Altaiar
Publisher: Packt Publishing Ltd
ISBN: 1800200285
Category : Computers
Languages : en
Pages : 185
Get Book Here
Book Description
Learn to extract actionable insights from your big data in real time using a range of Microsoft Azure features Key FeaturesUpdated with the latest features and new additions to Microsoft AzureMaster the fundamentals of cloud analytics using AzureLearn to use Azure Synapse Analytics (formerly known as Azure SQL Data Warehouse) to derive real-time customer insightsBook Description Cloud Analytics with Microsoft Azure serves as a comprehensive guide for big data analysis and processing using a range of Microsoft Azure features. This book covers everything you need to build your own data warehouse and learn numerous techniques to gain useful insights by analyzing big data The book begins by introducing you to the power of data with big data analytics, the Internet of Things (IoT), machine learning, artificial intelligence, and DataOps. You will learn about cloud-scale analytics and the services Microsoft Azure offers to empower businesses to discover insights. You will also be introduced to the new features and functionalities added to the modern data warehouse. Finally, you will look at two real-world business use cases to demonstrate high-level solutions using Microsoft Azure. The aim of these use cases will be to illustrate how real-time data can be analyzed in Azure to derive meaningful insights and make business decisions. You will learn to build an end-to-end analytics pipeline on the cloud with machine learning and deep learning concepts. By the end of this book, you will be proficient in analyzing large amounts of data with Azure and using it effectively to benefit your organization. What you will learnExplore the concepts of modern data warehouses and data pipelinesDiscover unique design considerations while applying a cloud analytics solutionDesign an end-to-end analytics pipeline on the cloudDifferentiate between structured, semi-structured, and unstructured dataChoose a cloud-based service for your data analytics solutionsUse Azure services to ingest, store, and analyze data of any scaleWho this book is for This book is designed to benefit software engineers, Azure developers, cloud consultants, and anyone who is keen to learn the process of deriving business insights from huge amounts of data using Azure. Though not necessary, a basic understanding of data analytics concepts such as data streaming, data types, the machine learning life cycle, and Docker containers will help you get the most out of the book.
Author: Richard L. Nuckolls
Publisher: Manning
ISBN: 1617296309
Category : Computers
Languages : en
Pages : 446
Get Book Here
Book Description
The Microsoft Azure cloud is an ideal platform for data-intensive applications. Designed for productivity, Azure provides pre-built services that make collection, storage, and analysis much easier to implement and manage. Azure Storage, Streaming, and Batch Analytics teaches you how to design a reliable, performant, and cost-effective data infrastructure in Azure by progressively building a complete working analytics system. Summary The Microsoft Azure cloud is an ideal platform for data-intensive applications. Designed for productivity, Azure provides pre-built services that make collection, storage, and analysis much easier to implement and manage. Azure Storage, Streaming, and Batch Analytics teaches you how to design a reliable, performant, and cost-effective data infrastructure in Azure by progressively building a complete working analytics system. Purchase of the print book includes a free eBook in PDF, Kindle, and ePub formats from Manning Publications. About the technology Microsoft Azure provides dozens of services that simplify storing and processing data. These services are secure, reliable, scalable, and cost efficient. About the book Azure Storage, Streaming, and Batch Analytics shows you how to build state-of-the-art data solutions with tools from the Microsoft Azure platform. Read along to construct a cloud-native data warehouse, adding features like real-time data processing. Based on the Lambda architecture for big data, the design uses scalable services such as Event Hubs, Stream Analytics, and SQL databases. Along the way, you’ll cover most of the topics needed to earn an Azure data engineering certification. What's inside Configuring Azure services for speed and cost Constructing data pipelines with Data Factory Choosing the right data storage methods About the reader For readers familiar with database management. Examples in C# and PowerShell. About the author Richard Nuckolls is a senior developer building big data analytics and reporting systems in Azure. Table of Contents 1 What is data engineering? 2 Building an analytics system in Azure 3 General storage with Azure Storage accounts 4 Azure Data Lake Storage 5 Message handling with Event Hubs 6 Real-time queries with Azure Stream Analytics 7 Batch queries with Azure Data Lake Analytics 8 U-SQL for complex analytics 9 Integrating with Azure Data Lake Analytics 10 Service integration with Azure Data Factory 11 Managed SQL with Azure SQL Database 12 Integrating Data Factory with SQL Database 13 Where to go next
Author: Zoiner Tejada
Publisher: "O'Reilly Media, Inc."
ISBN: 1491956623
Category : Computers
Languages : en
Pages : 411
Get Book Here
Book Description
Helps users understand the breadth of Azure services by organizing them into a reference framework they can use when crafting their own big-data analytics solution.
Author: Vlad Riscutia
Publisher: Simon and Schuster
ISBN: 1617298921
Category : Computers
Languages : en
Pages : 334
Get Book Here
Book Description
Build a data platform to the industry-leading standards set by Microsoft’s own infrastructure. Summary In Data Engineering on Azure you will learn how to: Pick the right Azure services for different data scenarios Manage data inventory Implement production quality data modeling, analytics, and machine learning workloads Handle data governance Using DevOps to increase reliability Ingesting, storing, and distributing data Apply best practices for compliance and access control Data Engineering on Azure reveals the data management patterns and techniques that support Microsoft’s own massive data infrastructure. Author Vlad Riscutia, a data engineer at Microsoft, teaches you to bring an engineering rigor to your data platform and ensure that your data prototypes function just as well under the pressures of production. You'll implement common data modeling patterns, stand up cloud-native data platforms on Azure, and get to grips with DevOps for both analytics and machine learning. Purchase of the print book includes a free eBook in PDF, Kindle, and ePub formats from Manning Publications. About the technology Build secure, stable data platforms that can scale to loads of any size. When a project moves from the lab into production, you need confidence that it can stand up to real-world challenges. This book teaches you to design and implement cloud-based data infrastructure that you can easily monitor, scale, and modify. About the book In Data Engineering on Azure you’ll learn the skills you need to build and maintain big data platforms in massive enterprises. This invaluable guide includes clear, practical guidance for setting up infrastructure, orchestration, workloads, and governance. As you go, you’ll set up efficient machine learning pipelines, and then master time-saving automation and DevOps solutions. The Azure-based examples are easy to reproduce on other cloud platforms. What's inside Data inventory and data governance Assure data quality, compliance, and distribution Build automated pipelines to increase reliability Ingest, store, and distribute data Production-quality data modeling, analytics, and machine learning About the reader For data engineers familiar with cloud computing and DevOps. About the author Vlad Riscutia is a software architect at Microsoft. Table of Contents 1 Introduction PART 1 INFRASTRUCTURE 2 Storage 3 DevOps 4 Orchestration PART 2 WORKLOADS 5 Processing 6 Analytics 7 Machine learning PART 3 GOVERNANCE 8 Metadata 9 Data quality 10 Compliance 11 Distributing data
Author: Ahmad Osama
Publisher: Packt Publishing Ltd
ISBN: 1800201540
Category : Computers
Languages : en
Pages : 455
Get Book Here
Book Description
Over 90 recipes to help you orchestrate modern ETL/ELT workflows and perform analytics using Azure services more easily Key FeaturesBuild highly efficient ETL pipelines using the Microsoft Azure Data servicesCreate and execute real-time processing solutions using Azure Databricks, Azure Stream Analytics, and Azure Data ExplorerDesign and execute batch processing solutions using Azure Data FactoryBook Description Data engineering is one of the faster growing job areas as Data Engineers are the ones who ensure that the data is extracted, provisioned and the data is of the highest quality for data analysis. This book uses various Azure services to implement and maintain infrastructure to extract data from multiple sources, and then transform and load it for data analysis. It takes you through different techniques for performing big data engineering using Microsoft Azure Data services. It begins by showing you how Azure Blob storage can be used for storing large amounts of unstructured data and how to use it for orchestrating a data workflow. You'll then work with different Cosmos DB APIs and Azure SQL Database. Moving on, you'll discover how to provision an Azure Synapse database and find out how to ingest and analyze data in Azure Synapse. As you advance, you'll cover the design and implementation of batch processing solutions using Azure Data Factory, and understand how to manage, maintain, and secure Azure Data Factory pipelines. You'll also design and implement batch processing solutions using Azure Databricks and then manage and secure Azure Databricks clusters and jobs. In the concluding chapters, you'll learn how to process streaming data using Azure Stream Analytics and Data Explorer. By the end of this Azure book, you'll have gained the knowledge you need to be able to orchestrate batch and real-time ETL workflows in Microsoft Azure. What you will learnUse Azure Blob storage for storing large amounts of unstructured dataPerform CRUD operations on the Cosmos Table APIImplement elastic pools and business continuity with Azure SQL DatabaseIngest and analyze data using Azure Synapse AnalyticsDevelop Data Factory data flows to extract data from multiple sourcesManage, maintain, and secure Azure Data Factory pipelinesProcess streaming data using Azure Stream Analytics and Data ExplorerWho this book is for This book is for Data Engineers, Database administrators, Database developers, and extract, load, transform (ETL) developers looking to build expertise in Azure Data engineering using a recipe-based approach. Technical architects and database architects with experience in designing data or ETL applications either on-premise or on any other cloud vendor who wants to learn Azure Data engineering concepts will also find this book useful. Prior knowledge of Azure fundamentals and data engineering concepts is needed.
Author: Michael Collier
Publisher: Microsoft Press
ISBN: 0735697302
Category : Computers
Languages : en
Pages : 400
Get Book Here
Book Description
Microsoft Azure Essentials from Microsoft Press is a series of free ebooks designed to help you advance your technical skills with Microsoft Azure. The first ebook in the series, Microsoft Azure Essentials: Fundamentals of Azure, introduces developers and IT professionals to the wide range of capabilities in Azure. The authors - both Microsoft MVPs in Azure - present both conceptual and how-to content for key areas, including: Azure Websites and Azure Cloud Services Azure Virtual Machines Azure Storage Azure Virtual Networks Databases Azure Active Directory Management tools Business scenarios Watch Microsoft Press’s blog and Twitter (@MicrosoftPress) to learn about other free ebooks in the “Microsoft Azure Essentials” series.
Author: Peter ter Braake
Publisher: Packt Publishing Ltd
ISBN: 1801076707
Category : Computers
Languages : en
Pages : 428
Get Book Here
Book Description
Choose the right Azure data service and correct model design for successful implementation of your data model with the help of this hands-on guide Key FeaturesDesign a cost-effective, performant, and scalable database in AzureChoose and implement the most suitable design for a databaseDiscover how your database can scale with growing data volumes, concurrent users, and query complexityBook Description Data is at the heart of all applications and forms the foundation of modern data-driven businesses. With the multitude of data-related use cases and the availability of different data services, choosing the right service and implementing the right design becomes paramount to successful implementation. Data Modeling for Azure Data Services starts with an introduction to databases, entity analysis, and normalizing data. The book then shows you how to design a NoSQL database for optimal performance and scalability and covers how to provision and implement Azure SQL DB, Azure Cosmos DB, and Azure Synapse SQL Pool. As you progress through the chapters, you'll learn about data analytics, Azure Data Lake, and Azure SQL Data Warehouse and explore dimensional modeling, data vault modeling, along with designing and implementing a Data Lake using Azure Storage. You'll also learn how to implement ETL with Azure Data Factory. By the end of this book, you'll have a solid understanding of which Azure data services are the best fit for your model and how to implement the best design for your solution. What you will learnModel relational database using normalization, dimensional, or Data Vault modelingProvision and implement Azure SQL DB and Azure Synapse SQL PoolsDiscover how to model a Data Lake and implement it using Azure StorageModel a NoSQL database and provision and implement an Azure Cosmos DBUse Azure Data Factory to implement ETL/ELT processesCreate a star schema model using dimensional modelingWho this book is for This book is for business intelligence developers and consultants who work on (modern) cloud data warehousing and design and implement databases. Beginner-level knowledge of cloud data management is expected.
Author: Dmitry Anoshin
Publisher: Packt Publishing Ltd
ISBN: 1800561024
Category : Computers
Languages : en
Pages : 383
Get Book Here
Book Description
Solve real-world data problems and create data-driven workflows for easy data movement and processing at scale with Azure Data Factory Key FeaturesLearn how to load and transform data from various sources, both on-premises and on cloudUse Azure Data Factory’s visual environment to build and manage hybrid ETL pipelinesDiscover how to prepare, transform, process, and enrich data to generate key insightsBook Description Azure Data Factory (ADF) is a modern data integration tool available on Microsoft Azure. This Azure Data Factory Cookbook helps you get up and running by showing you how to create and execute your first job in ADF. You’ll learn how to branch and chain activities, create custom activities, and schedule pipelines. This book will help you to discover the benefits of cloud data warehousing, Azure Synapse Analytics, and Azure Data Lake Gen2 Storage, which are frequently used for big data analytics. With practical recipes, you’ll learn how to actively engage with analytical tools from Azure Data Services and leverage your on-premise infrastructure with cloud-native tools to get relevant business insights. As you advance, you’ll be able to integrate the most commonly used Azure Services into ADF and understand how Azure services can be useful in designing ETL pipelines. The book will take you through the common errors that you may encounter while working with ADF and show you how to use the Azure portal to monitor pipelines. You’ll also understand error messages and resolve problems in connectors and data flows with the debugging capabilities of ADF. By the end of this book, you’ll be able to use ADF as the main ETL and orchestration tool for your data warehouse or data platform projects. What you will learnCreate an orchestration and transformation job in ADFDevelop, execute, and monitor data flows using Azure SynapseCreate big data pipelines using Azure Data Lake and ADFBuild a machine learning app with Apache Spark and ADFMigrate on-premises SSIS jobs to ADFIntegrate ADF with commonly used Azure services such as Azure ML, Azure Logic Apps, and Azure FunctionsRun big data compute jobs within HDInsight and Azure DatabricksCopy data from AWS S3 and Google Cloud Storage to Azure Storage using ADF's built-in connectorsWho this book is for This book is for ETL developers, data warehouse and ETL architects, software professionals, and anyone who wants to learn about the common and not-so-common challenges faced while developing traditional and hybrid ETL solutions using Microsoft's Azure Data Factory. You’ll also find this book useful if you are looking for recipes to improve or enhance your existing ETL pipelines. Basic knowledge of data warehousing is expected.
Author: Yuri Diogenes
Publisher: Microsoft Press
ISBN: 1509307060
Category : Computers
Languages : en
Pages : 298
Get Book Here
Book Description
Discover high-value Azure security insights, tips, and operational optimizations This book presents comprehensive Azure Security Center techniques for safeguarding cloud and hybrid environments. Leading Microsoft security and cloud experts Yuri Diogenes and Dr. Thomas Shinder show how to apply Azure Security Center’s full spectrum of features and capabilities to address protection, detection, and response in key operational scenarios. You’ll learn how to secure any Azure workload, and optimize virtually all facets of modern security, from policies and identity to incident response and risk management. Whatever your role in Azure security, you’ll learn how to save hours, days, or even weeks by solving problems in most efficient, reliable ways possible. Two of Microsoft’s leading cloud security experts show how to: • Assess the impact of cloud and hybrid environments on security, compliance, operations, data protection, and risk management • Master a new security paradigm for a world without traditional perimeters • Gain visibility and control to secure compute, network, storage, and application workloads • Incorporate Azure Security Center into your security operations center • Integrate Azure Security Center with Azure AD Identity Protection Center and third-party solutions • Adapt Azure Security Center’s built-in policies and definitions for your organization • Perform security assessments and implement Azure Security Center recommendations • Use incident response features to detect, investigate, and address threats • Create high-fidelity fusion alerts to focus attention on your most urgent security issues • Implement application whitelisting and just-in-time VM access • Monitor user behavior and access, and investigate compromised or misused credentials • Customize and perform operating system security baseline assessments • Leverage integrated threat intelligence to identify known bad actors
