Skip to content

feat(server): Enable upcoming integrations endpoint #40256

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 5 commits into from
Apr 16, 2025

Conversation

vivek-appsmith
Copy link
Contributor

@vivek-appsmith vivek-appsmith commented Apr 15, 2025

This PR adds a new endpoint to the Plugin API that fetches upcoming integrations from Appsmith's cloud services. This feature enables users to discover and prepare for new integration options that will be available in future releases.

Changes

  • Added a new getUpcomingIntegrations endpoint to the PluginController at GET /api/v1/plugins/upcoming-integrations
  • Implemented the corresponding service method in PluginService to fetch data from cloud services
  • Added proper error handling for cloud service communication issues
  • Injected CloudServicesConfig to ensure proper cloud service URLs are used
  • Added instanceId to all requests to cloud services for tracking and authorization

Technical Details

  • Uses WebClientUtils for HTTP requests to external services
  • Uses ConfigService to retrieve the instance ID for authentication
  • Handles error responses gracefully, returning empty results instead of errors to clients
  • Returns properly formatted integration data including names, descriptions, and other metadata

This enhancement will help users discover upcoming integration options and provide feedback on planned datasource connectors.

/ok-to-test tags="@tag.Sanity"

Solves #40048

Tip

🟢 🟢 🟢 All cypress tests have passed! 🎉 🎉 🎉
Workflow run: https://github.com/appsmithorg/appsmith/actions/runs/14490650726
Commit: a9ae874
Cypress dashboard.
Tags: @tag.Sanity
Spec:


Wed, 16 Apr 2025 11:50:36 UTC

Summary by CodeRabbit

  • New Features
    • Added a new endpoint to fetch upcoming integrations, providing users with information about future external service integrations.
  • Bug Fixes
    • Improved error handling for external integration data fetches, ensuring a graceful fallback and user-friendly messaging if data cannot be retrieved.

Copy link
Contributor

coderabbitai bot commented Apr 15, 2025

Walkthrough

The changes introduce a new REST endpoint in the PluginControllerCE class for fetching upcoming integration data from an external service. This involves injecting a new configuration dependency, CloudServicesConfig, into both the CE and main plugin controllers. The new endpoint uses a reactive WebClient to retrieve data, handles errors gracefully by returning a fallback response, and does not alter existing endpoints or logic. Additionally, the plugin service implementation and its interface were extended to support this new functionality, with necessary constructor updates for dependency injection. Corresponding test setup was also updated.

Changes

File(s) Change Summary
.../controllers/PluginController.java Updated constructor to accept and pass CloudServicesConfig as a new dependency.
.../controllers/ce/PluginControllerCE.java Added GET /upcoming-integrations endpoint: fetches data from an external API, handles errors, and returns results. Injected CloudServicesConfig.
.../plugins/base/PluginServiceCE.java Added new method getUpcomingIntegrations() to interface.
.../plugins/base/PluginServiceCEImpl.java Added dependencies CloudServicesConfig and ConfigService. Implemented getUpcomingIntegrations() to fetch data asynchronously from external API with error handling.
.../plugins/base/PluginServiceImpl.java Updated constructor to accept and pass CloudServicesConfig and ConfigService to superclass.
.../services/ce/PluginServiceCEImplTest.java Added fields and constructor parameters for CloudServicesConfig and ConfigService in test setup.

Sequence Diagram(s)

sequenceDiagram
    participant Client
    participant PluginControllerCE
    participant PluginServiceCEImpl
    participant ConfigService
    participant CloudServicesConfig
    participant ExternalAPI

    Client->>PluginControllerCE: GET /upcoming-integrations
    PluginControllerCE->>PluginServiceCEImpl: getUpcomingIntegrations()
    PluginServiceCEImpl->>ConfigService: getInstanceId()
    ConfigService-->>PluginServiceCEImpl: instanceId
    PluginServiceCEImpl->>CloudServicesConfig: getBaseUrl()
    CloudServicesConfig-->>PluginServiceCEImpl: baseUrl
    PluginServiceCEImpl->>ExternalAPI: GET baseUrl + "?instanceId=..."
    alt Success
        ExternalAPI-->>PluginServiceCEImpl: integration data
        PluginServiceCEImpl-->>PluginControllerCE: List<Map<String,String>>
        PluginControllerCE-->>Client: 200 OK with data
    else Error
        PluginServiceCEImpl-->>PluginControllerCE: empty list
        PluginControllerCE-->>Client: 200 OK with empty list and error message
    end
Loading

Poem

New endpoints bloom, a feature’s delight,
Fetching integrations from sources out of sight.
With configs injected and WebClient in tow,
Errors are handled—no panic, just flow.
The code grows more clever, the service more bright,
As plugins and cloud configs now join in their might.
🚀


📜 Recent review details

Configuration used: .coderabbit.yaml
Review profile: CHILL
Plan: Pro (Legacy)

📥 Commits

Reviewing files that changed from the base of the PR and between 139a5e5 and a9ae874.

📒 Files selected for processing (2)
  • app/server/appsmith-server/src/main/java/com/appsmith/server/controllers/ce/PluginControllerCE.java (4 hunks)
  • app/server/appsmith-server/src/main/java/com/appsmith/server/plugins/base/PluginServiceCEImpl.java (7 hunks)
🚧 Files skipped from review as they are similar to previous changes (2)
  • app/server/appsmith-server/src/main/java/com/appsmith/server/plugins/base/PluginServiceCEImpl.java
  • app/server/appsmith-server/src/main/java/com/appsmith/server/controllers/ce/PluginControllerCE.java
⏰ Context from checks skipped due to timeout of 90000ms (2)
  • GitHub Check: perform-test / build-docker-image / build-docker
  • GitHub Check: server-unit-tests / server-unit-tests

🪧 Tips

Chat

There are 3 ways to chat with CodeRabbit:

  • Review comments: Directly reply to a review comment made by CodeRabbit. Example:
    • I pushed a fix in commit <commit_id>, please review it.
    • Generate unit testing code for this file.
    • Open a follow-up GitHub issue for this discussion.
  • Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query. Examples:
    • @coderabbitai generate unit testing code for this file.
    • @coderabbitai modularize this function.
  • PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
    • @coderabbitai gather interesting stats about this repository and render them as a table. Additionally, render a pie chart showing the language distribution in the codebase.
    • @coderabbitai read src/utils.ts and generate unit testing code.
    • @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.
    • @coderabbitai help me debug CodeRabbit configuration file.

Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments.

CodeRabbit Commands (Invoked using PR comments)

  • @coderabbitai pause to pause the reviews on a PR.
  • @coderabbitai resume to resume the paused reviews.
  • @coderabbitai review to trigger an incremental review. This is useful when automatic reviews are disabled for the repository.
  • @coderabbitai full review to do a full review from scratch and review all the files again.
  • @coderabbitai summary to regenerate the summary of the PR.
  • @coderabbitai resolve resolve all the CodeRabbit review comments.
  • @coderabbitai configuration to show the current CodeRabbit configuration for the repository.
  • @coderabbitai help to get help.

Other keywords and placeholders

  • Add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.
  • Add @coderabbitai summary to generate the high-level summary at a specific location in the PR description.
  • Add @coderabbitai anywhere in the PR title to generate the title automatically.

Documentation and Community

  • Visit our Documentation for detailed information on how to use CodeRabbit.
  • Join our Discord Community to get help, request features, and share feedback.
  • Follow us on X/Twitter for updates and announcements.

@vivek-appsmith vivek-appsmith added the ok-to-test Required label for CI label Apr 15, 2025
@vivek-appsmith vivek-appsmith force-pushed the feature/enable-upcoming-integrations-endpoint branch from 524547d to 6a7a11d Compare April 15, 2025 13:01
Copy link

Failed server tests

  • com.appsmith.server.helpers.UserUtilsTest#makeInstanceAdministrator_WhenUserAlreadyAdmin_MaintainsPermissionsSuccessfully

@NilanshBansal NilanshBansal marked this pull request as ready for review April 16, 2025 05:56
Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 0

🧹 Nitpick comments (2)
app/server/appsmith-server/src/main/java/com/appsmith/server/controllers/ce/PluginControllerCE.java (2)

105-125: Consider enhancing error handling and type safety.

The implementation correctly fetches and handles upcoming integrations, but has room for improvement:

  1. Using a raw Map type reduces type safety. Consider creating a dedicated DTO class.
  2. API path should be extracted to a constant or configuration property.
  3. Consider adding a timeout for the external API call.
- @GetMapping("/upcoming-integrations")
- public Mono<ResponseDTO<Map>> getUpcomingIntegrations() {
+ private static final String UPCOMING_INTEGRATIONS_PATH = "/api/v1/external-saas/upcoming-integrations";
+ 
+ @GetMapping("/upcoming-integrations")
+ public Mono<ResponseDTO<Map<String, Object>>> getUpcomingIntegrations() {
    log.debug("Fetching upcoming integrations from external API");

-   String apiUrl = cloudServicesConfig.getBaseUrl() + "/api/v1/external-saas/upcoming-integrations";
+   String apiUrl = cloudServicesConfig.getBaseUrl() + UPCOMING_INTEGRATIONS_PATH;

    return WebClientUtils.create()
        .get()
        .uri(apiUrl)
+       .timeout(Duration.ofSeconds(5))
-       .retrieve()
-       .bodyToMono(Map.class)
-       .map(response -> new ResponseDTO<>(HttpStatus.OK, response))
+       .retrieve()
+       .bodyToMono(new ParameterizedTypeReference<Map<String, Object>>() {})
+       .map(response -> new ResponseDTO<>(HttpStatus.OK, response))
        .onErrorResume(error -> {
            log.warn("Error retrieving upcoming integrations from external service: {}", error.getMessage());
            return Mono.just(new ResponseDTO<>(
                    HttpStatus.OK.value(),
                    Map.of("integrations", List.of()),
                    "Unable to fetch upcoming integrations at this time"));
        });
}

Don't forget to add the missing imports:

import java.time.Duration;
import org.springframework.core.ParameterizedTypeReference;

119-123: Consider using appropriate HTTP status codes for error responses.

Currently, the error response returns HTTP 200 with an empty list. While this simplifies client handling, following RESTful conventions would suggest using an appropriate error status code.

One option is to use a more specific status code like 503 SERVICE_UNAVAILABLE when external services fail:

.onErrorResume(error -> {
    log.warn("Error retrieving upcoming integrations from external service: {}", error.getMessage());
    return Mono.just(new ResponseDTO<>(
-       HttpStatus.OK.value(),
+       HttpStatus.SERVICE_UNAVAILABLE.value(),
        Map.of("integrations", List.of()),
        "Unable to fetch upcoming integrations at this time"));
});

However, if the current approach is part of an established pattern in your application for handling errors, maintaining consistency is also important. Consider discussing with your team to establish consistent error handling policies.

📜 Review details

Configuration used: .coderabbit.yaml
Review profile: CHILL
Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between b39d21e and 6a7a11d.

📒 Files selected for processing (2)
  • app/server/appsmith-server/src/main/java/com/appsmith/server/controllers/PluginController.java (2 hunks)
  • app/server/appsmith-server/src/main/java/com/appsmith/server/controllers/ce/PluginControllerCE.java (4 hunks)
🔇 Additional comments (1)
app/server/appsmith-server/src/main/java/com/appsmith/server/controllers/PluginController.java (1)

3-3: Correctly updated constructor to support new endpoint.

The constructor properly injects the CloudServicesConfigCE dependency and passes it to the parent class, enabling the upcoming integrations endpoint functionality.

Also applies to: 15-19

NilanshBansal
NilanshBansal previously approved these changes Apr 16, 2025
Copy link
Contributor

@NilanshBansal NilanshBansal left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Copy link

Failed server tests

  • com.appsmith.server.helpers.UserUtilsTest#makeInstanceAdministrator_WhenUserAlreadyAdmin_MaintainsPermissionsSuccessfully

1 similar comment
Copy link

Failed server tests

  • com.appsmith.server.helpers.UserUtilsTest#makeInstanceAdministrator_WhenUserAlreadyAdmin_MaintainsPermissionsSuccessfully

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🧹 Nitpick comments (2)
app/server/appsmith-server/src/main/java/com/appsmith/server/plugins/base/PluginServiceCEImpl.java (2)

674-706: Implementation could use some improvements for robustness.

The method implementation works correctly but has a few areas that could be improved:

  1. Consider adding more comprehensive error logging
  2. The type casting from Object to Map<String, String> could be more robust
  3. Missing Javadoc documentation

Consider enhancing error handling and adding documentation:

 @Override
 public Mono<List<Map<String, String>>> getUpcomingIntegrations() {
     log.debug("Fetching upcoming integrations from external API");
 
     return configService.getInstanceId().flatMap(instanceId -> {
         String apiUrl = cloudServicesConfig.getBaseUrl()
                 + "/api/v1/config/external-saas/upcoming-integrations?instanceId=" + instanceId;
         return WebClientUtils.create()
                 .get()
                 .uri(apiUrl)
                 .retrieve()
                 .bodyToMono(Map.class)
                 .map(response -> {
                     // Extract the integrations list from the response
                     if (response.containsKey("data")) {
                         List<Map<String, String>> integrations = new ArrayList<>();
                         List<?> data = (List<?>) response.get("data");
                         for (Object item : data) {
-                            if (item instanceof Map) {
+                            if (item instanceof Map<?, ?> map) {
+                                try {
                                 integrations.add((Map<String, String>) item);
+                                } catch (ClassCastException e) {
+                                    log.warn("Invalid integration data format: {}", map);
+                                }
                             }
                         }
                         return integrations;
                     }
                     return List.<Map<String, String>>of();
                 })
                 .onErrorResume(error -> {
-                    log.warn(
-                            "Error retrieving upcoming integrations from external service: {}", error.getMessage());
+                    log.warn(
+                            "Error retrieving upcoming integrations from external service", error);
                     return Mono.just(List.<Map<String, String>>of());
                 });
     });
 }

Also, consider adding Javadoc:

/**
 * Fetches upcoming integrations from the external cloud service.
 * 
 * @return A Mono containing a list of integration details as Maps, or an empty list if the request fails
 */
@Override
public Mono<List<Map<String, String>>> getUpcomingIntegrations() {
    // Existing implementation...
}

679-680: Use URI builder for constructing URLs.

Concatenating strings to create URLs is error-prone. Consider using UriComponentsBuilder for more robust URL construction.

- String apiUrl = cloudServicesConfig.getBaseUrl()
-         + "/api/v1/config/external-saas/upcoming-integrations?instanceId=" + instanceId;
+ String apiUrl = UriComponentsBuilder.fromHttpUrl(cloudServicesConfig.getBaseUrl())
+         .path("/api/v1/config/external-saas/upcoming-integrations")
+         .queryParam("instanceId", instanceId)
+         .build()
+         .toString();

Remember to add the import:

import org.springframework.web.util.UriComponentsBuilder;
📜 Review details

Configuration used: .coderabbit.yaml
Review profile: CHILL
Plan: Pro (Legacy)

📥 Commits

Reviewing files that changed from the base of the PR and between 034bf9a and 139a5e5.

📒 Files selected for processing (5)
  • app/server/appsmith-server/src/main/java/com/appsmith/server/controllers/ce/PluginControllerCE.java (4 hunks)
  • app/server/appsmith-server/src/main/java/com/appsmith/server/plugins/base/PluginServiceCE.java (1 hunks)
  • app/server/appsmith-server/src/main/java/com/appsmith/server/plugins/base/PluginServiceCEImpl.java (7 hunks)
  • app/server/appsmith-server/src/main/java/com/appsmith/server/plugins/base/PluginServiceImpl.java (3 hunks)
  • app/server/appsmith-server/src/test/java/com/appsmith/server/services/ce/PluginServiceCEImplTest.java (3 hunks)
🚧 Files skipped from review as they are similar to previous changes (1)
  • app/server/appsmith-server/src/main/java/com/appsmith/server/controllers/ce/PluginControllerCE.java
🧰 Additional context used
🧬 Code Graph Analysis (1)
app/server/appsmith-server/src/main/java/com/appsmith/server/plugins/base/PluginServiceCEImpl.java (1)
app/client/src/ce/selectors/organizationSelectors.tsx (1)
  • getInstanceId (54-55)
⏰ Context from checks skipped due to timeout of 90000ms (5)
  • GitHub Check: perform-test / client-build / client-build
  • GitHub Check: perform-test / server-build / server-unit-tests
  • GitHub Check: perform-test / rts-build / build
  • GitHub Check: server-spotless / spotless-check
  • GitHub Check: server-unit-tests / server-unit-tests
🔇 Additional comments (3)
app/server/appsmith-server/src/main/java/com/appsmith/server/plugins/base/PluginServiceCE.java (1)

56-57: Interface addition looks good.

The new method signature follows the reactive programming pattern using Mono and returns a List of Maps, which is appropriate for a collection of integration details.

app/server/appsmith-server/src/main/java/com/appsmith/server/plugins/base/PluginServiceImpl.java (1)

3-3: Constructor updates are correctly propagated.

The changes properly pass the new dependencies to the parent class constructor.

Also applies to: 6-6, 28-30, 40-42

app/server/appsmith-server/src/main/java/com/appsmith/server/plugins/base/PluginServiceCEImpl.java (1)

5-5: Dependencies and imports correctly added.

The necessary imports and fields for the new functionality are properly included.

Also applies to: 19-19, 21-21, 52-52, 72-72, 94-94, 106-107, 114-115

Copy link

Failed server tests

  • com.appsmith.server.helpers.UserUtilsTest#makeInstanceAdministrator_WhenUserAlreadyAdmin_MaintainsPermissionsSuccessfully

@NilanshBansal NilanshBansal self-requested a review April 16, 2025 10:57
Copy link

Failed server tests

  • com.appsmith.server.helpers.UserUtilsTest#makeInstanceAdministrator_WhenUserAlreadyAdmin_MaintainsPermissionsSuccessfully

@vivek-appsmith vivek-appsmith enabled auto-merge (squash) April 16, 2025 11:37
@NilanshBansal NilanshBansal disabled auto-merge April 16, 2025 11:43
@vivek-appsmith
Copy link
Contributor Author

/build-deploy-preview skip-tests=true

Copy link

Deploying Your Preview: https://github.com/appsmithorg/appsmith/actions/runs/14492016386.
Workflow: On demand build Docker image and deploy preview.
skip-tests: true.
env: ``.
PR: 40256.
recreate: .

@NilanshBansal NilanshBansal merged commit 99e3cb5 into release Apr 16, 2025
45 checks passed
@NilanshBansal NilanshBansal deleted the feature/enable-upcoming-integrations-endpoint branch April 16, 2025 11:57
Copy link

Deploy-Preview-URL: https://ce-40256.dp.appsmith.com

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
ok-to-test Required label for CI
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants