An agency in Alabama is seeking information from vendors to identify a big data extraction tool capable of supporting secure, scalable data processing and analysis. The objective is to find technologies and solutions that enable efficient extraction, transformation, and analysis of data at scale within an architecture that accommodates unclassified, classified, and sensitive data. The agency’s data lake architecture requires layered access controls, robust governance, and adherence to legal and policy guidelines for workflow management.

Vendors are requested to provide information on software solutions that are available in both IL 5 and IL 6 environments. The agency is interested in distributed big data processing frameworks, and solutions should feature an interactive and intuitive development interface to facilitate rapid code development, bug triage, and data visualization. Additional requirements include compute cluster management and a description of any built-in or compatible Artificial Intelligence (AI) capabilities that the solution offers.