Short answer: A BI tool is different from most SaaS. It holds credentials to your databases, runs queries against your most sensitive tables, and shows the results to people across your organization and sometimes outside it. So the standard SaaS questionnaire misses the questions that matter most: how the tool connects to your data, whether it copies it, how it decides who sees which rows, and where the audit trail lives. Below are 43 questions grouped into 13 topics, with the answer that should satisfy a reviewer and the answer that should worry one. Use it as your questionnaire, or as a checklist against the vendor's.
Why BI needs its own questionnaire
Most vendor security questionnaires were written for tools that hold a slice of your data: a CRM, a ticketing system, a file share. A BI tool is closer to a database client with a web interface. It stores connection credentials for your warehouse, ERP, or clinical system. It executes SQL against them on demand. It renders results to users whose only qualification is a login, and it may email those results out on a schedule. A weakness anywhere in that chain, from a credential in plaintext to a row filter a user can remove, exposes the source data, not just the tool's own records.
That is why the questions below spend as much time on data handling and access design as on the vendor's own controls. It is also why the answers matter more than certifications. SOC 2 tells you a vendor has controls over its own environment; it says nothing about whether the product can leak one customer's rows to another user.
The 43 questions
Each topic maps to a section of our own security review kit, where we answer these questions for DashboardFox. Where we have a weak spot, it is stated there too.
1. Deployment and tenancy
| Question | A good answer | A warning sign |
| Is the service multi-tenant, single-tenant, or available both ways? | Both, with a clear description of what isolation each provides, and single-tenant available without an enterprise contract | "Multi-tenant, but it's secure" with no detail |
| In the multi-tenant case, what separates customers: separate databases, separate schemas, or a tenant ID column? | Separate database per customer with unique credentials | A tenant ID column in shared tables, which puts every customer one query bug away from another's data |
| Can we choose the region where our data and metadata are stored? | Yes, named regions, and data stays there | "We use global infrastructure" |
| Is a self-hosted option available if our requirements change? | Yes, same product, on your infrastructure | Cloud only, or a self-hosted version that lags the cloud product by years |
2. Hosting and sub-processors
| Question | A good answer | A warning sign |
| Who hosts the service, and where are its attestations? | A named provider with published SOC 2 or ISO 27001 reports, and the vendor knows which facility | The vendor cannot name the data center or provider |
| Which sub-processors can access our data, and is the list published? | A short published list with what each one does; every one that stores data is under a DPA or BAA | A long list, or one that includes analytics and AI providers the vendor did not mention |
| Will you notify us before adding a sub-processor? | Yes, with a stated notice period | Silence, or "see our terms" |
3. How the tool handles our data
| Question | A good answer | A warning sign |
| Does the tool copy our data into its own storage, or query our database in place? | Queries in place; holds only report definitions and metadata | Requires importing your data into the vendor's store to work, which doubles your breach surface |
| What is cached, for how long, and is the cache encrypted? | A specific answer with a retention period | "Results are cached for performance" with no detail |
| Are database credentials stored encrypted, and who can retrieve them? | Encrypted secrets store; retrieval requires elevated, logged access | Credentials visible to support staff, or stored in configuration files |
| Do operational logs contain our report data or personal data? | No; logs are configured to exclude customer data | The vendor has not thought about it |
| Do any AI or machine-learning features process our data, and where does it go? | A clear yes or no, naming the model provider if yes, with an opt-out | AI features enabled by default that send prompts and data to a third party not on the sub-processor list |
4. Encryption and secrets
| Question | A good answer | A warning sign |
| How is data encrypted at rest and in transit? | AES-256 at rest, TLS 1.2 or higher in transit, applied to every database and backup | Encryption only for "sensitive fields," or the vendor cannot say which version of TLS |
| Are backups encrypted before they leave the server, and who holds the key? | Encrypted client-side; key held offline by the vendor, not by the backup provider | Backups encrypted only by the storage provider, which means the provider can read them |
| Are scheduled reports sent through our mail system or yours? | Through your SMTP, so report data never transits shared mail infrastructure | Only through the vendor's shared mail relay |
5. Workforce access
| Question | A good answer | A warning sign |
| How many people can access our environment, and how? | A small number of named people; key-based access to hardened hosts; no contractors | Unknown, or a large support team with shared credentials |
| How is elevated access to our data granted and recorded? | Just-in-time, with a logged reason, and credentials rotated after use | A standing admin account that support uses when needed |
| Are staff background-checked, and is access reviewed and revoked on departure? | Yes, with a review cadence | No policy |
6. Authentication and authorization in the product
| Question | A good answer | A warning sign |
| What password policy is enforced, and is it configurable? | A minimum length, breached-password screening, and lockout, aligned to NIST 800-63B | Eight characters and no lockout |
| Is multi-factor authentication available for end users, and can we require it? | Yes, and it can be enforced | MFA for admins only |
| Do you support SSO with our identity provider? | SAML or OIDC, or an honest statement of what is available now and what is in development | "Yes" that turns out to mean a shared password page |
| How does row-level security work, and can a user remove it? | A server-side policy applied to every query for that user; nothing on the report side can bypass it | Per-report filters, URL parameters, or anything the vendor's own documentation says not to use for security |
| Can we restrict access by IP, and control session length and concurrency? | Yes to all three | None available below an enterprise tier |
7. Logging and audit
| Question | A good answer | A warning sign |
| What does the audit trail record? | Logins, report executions, data source access, permission changes, exports, and prints | Logins only |
| Where is the audit trail stored, and can we query it ourselves? | In your own database, queryable in the product, not ingested into the vendor's systems | Only available by asking support |
| How long are logs retained? | A stated period that meets your regulatory need (six years for HIPAA) | Thirty days |
8. Security testing and vulnerability management
| Question | A good answer | A warning sign |
| What penetration testing do you do, by whom, how often, and can we see results? | Recurring third-party testing that covers cross-tenant access control, with results shared under NDA | "We have a security team" and no report |
| Do you have a published vulnerability disclosure policy? | Yes, with response times | No way for a researcher to report a flaw |
| How quickly are critical vulnerabilities patched? | Emergency maintenance for critical fixes; automatic OS patching | A quarterly release cycle with no exception process |
9. Change management and secure development
| Question | A good answer | A warning sign |
| Is every production change reviewed by someone other than its author and tied to a record? | Yes, pull-request review and ticket traceability | Developers deploy directly |
| Are development and production separated, and are secrets kept out of code? | Yes and yes | Shared environments, or a past incident involving a leaked key |
10. Incident response and breach notification
| Question | A good answer | A warning sign |
| Is there a written incident response plan, and how fast will you tell us about an incident? | A documented plan with named roles; acknowledgement within hours and updates on a stated cadence | "We'll let you know" |
| What is your breach notification commitment? | A contractual period, such as 72 hours, that lets you meet your own deadlines | Nothing in writing |
11. Business continuity and disaster recovery
| Question | A good answer | A warning sign |
| How often are backups taken, how long are they kept, and where? | A schedule, a retention period, and an offsite location in your region | The vendor has to check |
| When did you last test a restore? | A date within the last year, with results recorded | Never, or "backups are automatic" |
| What is your uptime commitment, and how is downtime credited? | A published SLA with a percentage and a credit | "Best effort" |
| What happens if your company goes out of business? | An honest answer: data export on demand, and ideally a self-hosted edition you could run yourself | No plan |
12. Physical and endpoint security
| Question | A good answer | A warning sign |
| Who is responsible for physical security of production, and can we see their attestation? | The hosting provider, with reports available | The vendor runs servers in an office |
| Are staff workstations encrypted, patched, and protected? | Full-disk encryption, automatic updates, host firewall, anti-malware, documented standard | No standard |
13. Compliance and insurance
| Question | A good answer | A warning sign |
| Which agreements will you sign: DPA, BAA, FERPA addendum, others? | A list, with the plans on which each is available | "We're compliant with everything" |
| Do you hold SOC 2, and if not, what do you offer instead? | Either a current report, or a documented policy set mapped to questionnaire items with a stated plan and date for SOC 2 | No report and no documentation |
| What cyber liability insurance do you carry, and can we see the certificate? | A specific limit and a certificate on request | The vendor does not know |
| What happens to our data at termination? | Return or deletion on a stated schedule, backups purged | Unstated |
Reading the answers
Three patterns separate vendors quickly.
Specificity. Good answers contain numbers, names, and dates: TLS 1.2, six years, tested in March, two named engineers. Weak answers contain adjectives: robust, industry-standard, enterprise-grade. A vendor that answers "how long are logs kept?" with "we take security very seriously" has told you the answer is "we don't know."
Admitted gaps. No small vendor has everything. A vendor that says "native SAML is in development; here is what we offer today" is telling you the truth about one thing, which makes the rest of its answers more credible. A vendor that claims a clean sheet is either very large or not reading the questions.
Documents on request. Ask for the incident response plan, not a summary of it. Ask for the last penetration test report under NDA. Ask for the certificate of insurance. Vendors that have done the work can produce these in a day. Vendors that have not will stall, and the stall is your answer.
How DashboardFox answers these
Every question above is answered for DashboardFox on our security review kit page, in the same 13 topics, with links to the underlying policy where one is published. The short version: shared, dedicated, and self-hosted deployment; OVHcloud hosting in US and EU regions with a published sub-processor list; queries in place with no copying of your data; AES-256 at rest and TLS 1.2 or higher in transit; backups encrypted before they leave the server with the key held offline; production access limited to a small number of named people over key-based connections with just-in-time, logged elevated credentials; NIST 800-63B password policy with breach screening and lockout; server-enforced row-level and field-level security on every plan; an audit trail that lives in your own database and is not ingested into ours; continuous third-party penetration testing with results under NDA; a published vulnerability disclosure policy; documented incident response, continuity, and change management; 72-hour breach notification; a 99.5% uptime SLA; and a BAA available on Enterprise Dedicated Cloud, backed by a standalone $5 million cyber liability policy.
The gaps, since a questionnaire should surface them: SOC 2 Type II is on our roadmap and not yet complete, and our controls are documented instead in an 18-policy set with a traceability matrix that maps each control to standard questionnaire items. Native SAML and OIDC single sign-on for end users is in development; today end users get session-token SSO plus optional MFA through Cisco Duo, and administrators get SSO through an identity-aware proxy. We would rather you learn both of those here than in the middle of a review. We have passed enterprise security reviews on this documentation, including one for a healthcare organization that now runs PHI on DashboardFox under a signed BAA.
Open the security review kit → · Download the security overview (PDF) → · Request the full security pack →
Frequently asked questions
Does a BI vendor need SOC 2 to pass a security review?
No. SOC 2 is a convenient package of evidence, and many reviewers ask for it first, but the requirement is the controls, not the report. A vendor without SOC 2 can pass with documented policies, test results, and honest answers; a vendor with SOC 2 can still fail on product-level questions like row-level security or credential storage, which SOC 2 does not examine.
What is the most important question to ask a BI vendor?
Whether the tool copies your data into its own storage or queries it in place. Everything else about your exposure follows from that answer: how many copies exist, where they are, how they are encrypted, and what has to be deleted when you leave.
How is row-level security supposed to work in a BI tool?
As a policy applied by the server to every query a user runs, based on who they are, with nothing on the report or dashboard side able to remove it. Per-report filters, URL parameters, and separate copies of dashboards per audience are not security controls, and several vendors' own documentation says so.
Should we send our standard SaaS questionnaire or a BI-specific one?
Send your standard one for the vendor's own controls, and add the product questions from topics 1, 3, 6, and 7 above. Those are the ones generic questionnaires miss, and they are where BI tools differ most.
How long should a BI vendor security review take?
With a vendor that has its documentation ready, one to two weeks including a call. If it is taking longer, the delay is usually the vendor assembling answers for the first time, which is itself useful information.
This is a practitioner's checklist, not legal advice. Your security and compliance teams decide what your organization requires.
